Product handling and packaging system

ABSTRACT

Embodiments of product handling systems facilitate transfer of individual product items from incoming bulk form into dedicated trays for inspection, sorting, selection, and packaging. Inspection may comprise interrogation of product items within a tray by electromagnetic (e.g., optical, hyperspectral) or other (e.g., physical, acoustic, gas sensing, etc.) techniques. Prior to packaging, product items disposed within the tray may be stored in a moveable carousel responsible for controlling environmental factors such as temperature, humidity, illumination, ambient gases, product-to-product interactions, and/or others. Movement of product items from a carousel&#39;s transfer station to an outside staging position may be accomplished using robots and/or conveyor belts. Embodiments may allow rapid, low-cost consumer selection of specific individual product items based upon their accompanying metadata (e.g., source, identifier), in combination with the results of inspection (e.g., visual appearance). Embodiments may receive product items pre-packaged in tray format to expedite inspection, sorting, selection, and packaging.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation-in-Part (CIP) of U.S. Nonprovisionalpatent application Ser. No. 16/189,673 filed Nov. 13, 2018 (issued asU.S. Pat. No. 10,543,942), which claims priority to U.S. ProvisionalPatent Application No. 62/589,409 filed Nov. 21, 2017, and also to U.S.Provisional Patent Application No. 62/675,656, filed on May 23, 2018,which applications are incorporated entirely herein by reference for allpurposes.

BACKGROUND

Efficient handling of many different types of items that exhibit avariety of shapes and/or sizes, can pose an increasingly complextechnological challenge. For example, produce items alone offered by aconventional grocery store, may exhibit sizes ranging from that of araisin to that of a watermelon. Moreover, the quality of such produceitems can degrade over time, affecting their monetary value.

SUMMARY

Product handling systems according to embodiments facilitate transfer ofindividual product items from incoming bulk form into dedicated traysfor subsequent inspection, sorting, selection, and packaging forconsumption. Inspection may comprise interrogation of product itemswithin a tray by electromagnetic (e.g., optical, hyperspectral) or other(e.g., physical, acoustic, gas sensing, etc.) techniques. Prior topackaging, product items disposed within the tray may be stored in amoveable carousel that is responsible for controlling environmentalfactors such as temperature, humidity, illumination, ambient gases,product-to-product interactions, and/or others. Movement of productitems from a carousel's transfer station to an outside staging positionmay be accomplished using robots and/or conveyor belts. Embodiments mayallow rapid, low-cost consumer selection of specific individual productitems based upon their accompanying metadata (e.g., source, identifier),in combination with the results of inspection (e.g., visual appearance).Some embodiments may receive product items that are already pre-packagedin tray format in order to expedite inspection, sorting, selection, andpackaging.

An embodiment of an apparatus comprises a frame translatable to astaging position in front of a carousel, and a fork supported by theframe and configured to extend in a direction toward the carousel toengage a tray holding an item. The apparatus further comprises amoveable member configured to project vertically into a first opening ofthe tray, contact the item at a first point, and to dispense the itemfrom the tray in a direction away from the carousel, into packaging.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified isometric view of a product handling systemaccording to an embodiment.

FIG. 1A is simplified side view of the embodiment of FIG. 1.

FIG. 1B is a simplified plan view of the embodiment of FIG. 1.

FIG. 1C is a simplified enlarged isometric view of the embodiment ofFIG. 1 illustrating six carousels.

FIG. 1D is another simplified plan view including a cut-away, of theembodiment of FIG. 1.

FIG. 2 is simplified perspective view illustrating a tray according toan embodiment. FIGS. 2A-2B show edge and plan views, respectively.

FIG. 3 is simplified plan view illustrating an embodiment of a trayhaving groove features configured to hold items.

FIG. 4 is simplified plan view graphically illustrating a product itemdisposed on a tray by a conveyor.

FIGS. 5A-D are various simplified views illustrating a carouselaccording to an embodiment.

FIG. 6 shows a perspective view of a tray with product items disposedtherein, interacting with a transfer mechanism. FIGS. 6A-B show plan andedge views, respectively.

FIGS. 7A-C are simplified side views illustrating operation of atransfer mechanism according to one embodiment.

FIG. 8 shows a simplified side view illustrating operation of a transfermechanism featuring flaps according to an embodiment.

FIG. 9 shows a simplified side view illustrating operation of a transfermechanism according to another embodiment.

FIG. 10 shows a simplified flow diagram of a method according to anembodiment.

FIG. 11 is a simplified diagram illustrating an embodiment of a producthandling system according to an example.

FIG. 12 shows a perspective view of an embodiment of a product handlingsystem.

FIG. 13 shows a side view of one carousel bank of the system of FIG. 12.

FIG. 14 shows a perspective view of a bag approaching and halting infront of the carousel on the transport path.

FIG. 15 shows a perspective view of forks from the frame extending intothe carousel and lifting a tray.

FIG. 16 shows a perspective view of the frame showing the forksextended.

FIG. 17 shows an end view of the frame showing the fork extendingunderneath the edge of the tray loaded with products

FIG. 18 shows a perspective view of the frame prior to the extension ofthe forks.

FIGS. 19A-D show enlarged views of the front of the frame during theretrieval process.

FIG. 20 is an end view of the frame with the extended forks engaging thetray as part of the retrieval process.

FIG. 21 shows a perspective view of a product conveyor which includes aslide.

FIG. 22 shows a perspective view of a tray (here empty for illustration)lowering over the product conveyors.

FIG. 23 shows a perspective view of the product conveyors engaging tolift the products from the lowered tray.

FIGS. 24A-B show side views of the product conveyor in non-extended andextended positions, respectively.

FIG. 25 shows a side view of the extended product conveyor bearingproducts lifted from the tray.

FIG. 26 shows a front view of the item being moved off of the extendedproduct conveyor into the bag.

FIGS. 27A-C are side views showing a sequence of disposing a productinto a bag according to an embodiment.

FIGS. 28-28B show views of a first tray type according to an embodiment.

FIGS. 29-29B show views of a second tray type according to anembodiment.

FIG. 30 shows a simplified view of an approach taking a series ofoptical images of a product (here a bunch of bananas) at different stageof its freshness lifetime.

FIG. 31 is a histogram of saturation data.

FIG. 32 shows a simplified flow diagram illustrating a machine learningprocess.

FIG. 33 shows a simplified diagram illustrating the implementation ofartificial intelligence principles to product handling according toembodiments.

FIGS. 34A-B show NIR images, and FIGS. 34C-D show RGB images, resultingfrom camera inspection.

FIG. 35 shows a simplified spectrograph.

FIGS. 36A-B show perspective views of a single conveyor having a linkagethat can be angled from flat over a range of angles.

FIGS. 37A-F show simplified views of pivoting belt conveyor flows.

FIGS. 38A-D show various views of an embodiment of a simple convey withmultiple cameras approach.

FIGS. 39A-B show different views of an approach imaging items in freefall.

FIG. 40 shows a side view of a star wheel conveyor according to anembodiment.

FIGS. 40A-C show simplified views illustrating that spheres andcylinders can be rotated as the conveyor belt runs. FIGS. 40D-F showsimplified views of the flipping of cuboid items when going through thesystem.

FIG. 41A shows a top view, and FIGS. 41B-C show end views, of a popuproller conveyor.

FIG. 42A shows a top view, and FIGS. 42B-C show perspective end views ofan embodiment of a roller and spinner conveyor mechanism.

FIGS. 43A-B illustrate top and side views, respectively, of a popthrough conveyor mechanism that may be utilized to position items on atray.

FIGS. 44A and 44B show simplified top and side views respectively of aXYZ gantry mechanism according to an embodiment that may be utilized toposition items on a tray.

FIG. 45A shows a side view, and FIGS. 45B-C show top views, illustratinga vertical stack buffer system according to an embodiment that may beutilized to position items on a tray.

FIG. 46 shows a simplified perspective view of a carousel frontaccording to an embodiment.

FIG. 46A illustrates a perspective view of one possible embodiment of acarrier.

FIG. 46B illustrates a perspective view of an alternative embodiment ofa carrier.

FIG. 46C shows a simplified front view of a carrier.

FIG. 46D shows carriers having a different number of levels, andper-level pitches.

FIG. 47 shows a perspective view of a front side of a carouselembodiment having a dispensing station attached thereto.

FIG. 48A shows an enlarged view of a tail on a conveyor. FIG. 48B showsa simplified perspective view of the pop-through conveyor of thedispense station, poised to dispense item(s) from a particular tray row.

FIG. 49A shows a side view of a pop-up conveyor mechanism including anoptical sensor for dispensed item detection. FIG. 49B shows an enlargedview of the optical sensor in the pop-up conveyor.

FIG. 50 shows a perspective view of a dispense station configured withload cells.

FIG. 51A shows a tray arriving at the carousel. FIG. 51B shows the trayloaded into the carousel.

FIG. 51C shows a simplified view illustrating the use of indexing pins.

FIG. 52A illustrates a simplified side view of an end effector designaccording to one embodiment. FIGS. 52B-C show alternative embodiments.

FIGS. 53A-B show different views of embodiments of a carrier design.

FIGS. 54A-B show perspective views of embodiments of a traveler.

FIGS. 55A-B show views of a traveler embodiment incorporating loadcells.

FIGS. 56A-B show top perspective, and enlarged views respectively, of atraveler including a frame.

FIGS. 57A-B shows different embodiments of travelers.

FIGS. 58A-B show simplified views of a plunger and grommet, and travelerembodiment incorporating same.

FIGS. 59A-C show simplified views of a hinged lid.

FIGS. 60A-B are perspective views of a traveler with the lid open andclosed, respectively.

FIGS. 61A-B are perspective views of different battery pack embodiments.

FIG. 62 shows conveyors that can stop at a specific stage, in order toreceive items dispensed from a given carousel.

FIG. 63A shows an end view of a tray that is configured to hold circularitems.

FIG. 63B shows an end view of a tray configured to hold cuboid items.

FIG. 64 shows a top view of a tray according to an embodiment.

FIGS. 65A-B show perspective views of multiple tray types that may beassembled from a plurality of parts.

FIGS. 66A-B are enlarged tray views showing multiple types of supportbeams.

FIGS. 67A-B are enlarged views showing tray features formed frommultiple molds.

FIG. 68 shows a simplified view of a gantry robot according to anembodiment.

FIG. 69 is a simplified flow diagram summarizing process controlaccording to an embodiment.

FIG. 70 shows a simplified view of food item categories organized into atree hierarchy for searching.

FIG. 71 shows an embodiment of a computer system utilized to implementitem handling.

FIG. 71A illustrates basic subsystems in the computer system of FIG. 71.

FIGS. 72A-B show views of different traveler embodiments featuring adrive system.

FIGS. 73A-B show embodiments of tracked and wheeled travelers,respectively, featuring multiple bags added to a single drive unit.

FIGS. 74A-B show embodiments of tracked and wheeled travelers,respectively, carrying a tray.

FIGS. 75A-C show respective front, front perspective, and sideperspective views of a wheeled tray traveler robot.

FIG. 76 is a simplified block diagram showing an overview of a systemaccording to an embodiment.

FIGS. 77A-E show various views of a design for an inspection station.

FIGS. 78A-C show various views of a carrier design.

FIGS. 79A-B show various views of a foam attachment approach.

FIGS. 80A-B show various views of an embodiment of a dispensingapproach.

FIGS. 81A-C show various views of an embodiment of a dispensingapproach.

FIG. 82 shows a side view illustrating an embodiment of a dispensingapproach.

FIG. 83 shows a top view illustrating an embodiment of a dispensingapproach.

FIG. 84 shows a simplified cross-sectional view of a product handlingapparatus comprising a moving carousel.

FIGS. 85A-85F are simplified cross-sectional views showing variousembodiments of a product handling system comprising fixed shelving.

FIG. 86 shows a simplified cross-sectional view of an embodiment of acentral lift system having one or more lift mechanisms in the same liftshaft.

FIG. 87A shows a top view of a repeating shelving design. FIG. 87B showsa perspective view of one unit in a repeating shelving design. FIG. 87Cshows an enlarged top view.

FIGS. 88A-88F show simplified side elevational views of center traydispensing in operation.

FIGS. 89A-89F show corresponding top views of retrieving a particularitem from a tray center.

FIG. 90 shows a simplified cross-section of a configuration in whichtrays are removed and returned to fixed locations utilizing a traytransport mechanism.

FIGS. 91A-B show views of tray loading shuttle conveyors for mechanicalinspection design, in operation.

FIG. 92 shows a simplified top view of spiral rollers for mechanicalinspection, in operation.

FIG. 93 shows an example of an embodiment of an inspection inductionsystem loading stations.

FIG. 94A shows a circular V-belt loading station in loading operation.FIG. 94B shows the circular V-belt loading station in unloadingoperation.

FIG. 95A shows a translating shuttle buffer in loading operation. FIG.95B shows the translating shuttle buffer in unloading operation.

FIG. 96A shows a stacked shuttle buffer in loading operation. FIG. 96Bshows the stacked shuttle buffer in unloading operation.

FIG. 97A is a simplified overhead view showing cases of product loadedonto an infeed conveyor and transferred to a 2D gantry conveyor. FIG.97B is a simplified side view showing product cases transferred from abay of shelving units back to the gantry conveyor.

FIG. 98A is a simplified top view of a tray design that allows dual sidehandling.

FIG. 98B is an enlarged perspective view of the tray of FIG. 98A.

FIGS. 99A-C show perspective, side, and cross-sectional views of anattachment design according to an embodiment.

FIG. 100A shows a perspective view of a dispense station according to anembodiment. FIG. 100B shows an enlarged view of the dispense station.

FIG. 100C shows an enlarged view of a tray picking mechanism.

FIG. 100D shows an enlarged view of a product dispensing mechanism in alowered position. FIG. 100E shows an enlarged view of the productdispensing mechanism in a raised position.

FIG. 100F shows independent carriage actuation accomplished utilizing amotor anchored to the moving carriage that includes the conveyor belt.

FIG. 100G shows an overhead perspective view of an alternativeembodiment of a product dispensing mechanism.

FIG. 101A shows an overhead view of a dispense mechanism located betweena carousel and a plurality of tray rovers. FIG. 101B shows a perspectiveview of movement of the dispense mechanism in a first direction. FIG.101C shows a perspective view of the movement of the dispense mechanismin a second, opposite direction. FIGS. 101D-F show enlarged views of abidirectional arm and hook in operation.

FIG. 102A shows a front perspective view of an elevator design. FIG.102B shows a side view of the elevator.

FIG. 103A shows a top view of an alternative embodiment of a traydesign. FIG. 103 shows an enlarged perspective view of the tray designof FIG. 103B. FIG. 103C shows an enlarged underside perspective view ofthe tray of FIG. 103A being engaged.

FIG. 104A shows a perspective view of an embodiment of tray loadingshuttle conveyors. FIG. 104B shows an overhead view of the tray loadingshuttle conveyors of FIG. 104A.

FIGS. 105A-B show overhead views of inspection size detectionadjustment.

FIG. 106 shows a perspective view of an alternative tray embodiment.

FIGS. 107A-B show overhead images of customer carrier packing withstereo depth vision.

FIG. 108 shows a side perspective view of a bag holder apparatusaccording to an embodiment.

FIG. 109 shows a simplified overhead perspective of the mechanism for amoving base of a bag holder apparatus.

FIG. 110 shows a tray stack.

FIG. 111 shows a simplified view of a locking feature.

FIG. 112 shows a simplified front view of an alignment feature.

FIG. 113A shows a side perspective view of a lift located within a shaftand in the process of retrieving a tray from fixed shelving.

FIG. 113B shows the lift holding the tray descending over the conveyorsof the dispense station. FIG. 113C shows the conveyors independentlymoving to align to the width of the slots in the tray. FIG. 113D showsthe elevator continuing to descend.

FIG. 114 is a simplified block diagram showing inputs to a computersystem comprising a processor and a database, and corresponding outputs.

DESCRIPTION

FIG. 1 is a simplified isometric view of a product handling systemaccording to an embodiment. FIG. 1A is simplified side view of theembodiment of FIG. 1.

In particular, product handling system 100 comprises an initial sorting,inspecting and tray loading areas 102 that are configured to receivebulk items. This input region 102 is in turn in communication with trayconveyor network 104.

In this particular embodiment, the tray conveyor network is locatedabove a second customer conveyor network 106. However, this is notrequired and the relative vertical locations of the first and secondconveyor networks could be reversed, or in accordance with still furtheralternative embodiments, the two (tray, customer) conveyor networkscould be positioned at a same vertical level. According to yet otherembodiments, the separate customer conveyor network that deliverspackages for output, can be located on an opposing side as the incomingtray conveyor network.

Trays are fed along the tray conveyor to the input area from a traybuffer 108. The tray buffer in turn receives the trays from a traywasher station 110 that functions to sanitize the trays once they havecompleted their previous product handling activities.

As further shown in FIG. 1, once product items are disposed onto thetrays, the trays move along the tray conveyor belt from the input regionto the carousels 112. There, the trays are loaded (e.g., by a robotand/or conveyor 113) into the carousels for storage under controlledconditions.

As previously mentioned, the product handling system of FIG. 1 furthercomprises a second, customer conveyor network 106 that is located at alower level relative to the upper, tray conveyor network. Thisparticular embodiment shows the customer conveyor network 106 as adouble lane configuration.

For particular embodiments utilizing conveyor belts, those conveyorbelts may cause fixed separation between transported trays and/orpackaged product items. The belts may be imaged to determine cleaningneeds. A conveyor belt may be configured to lift layer-by-layer, and mayfeature grooves and or duster-like lifters. Certain embodiments mayprovide a human-assisted station sharing a conveyor with robot lifters.

Via the customer conveyor network, the customer packaging deploymenttool 114 provides empty product packaging (e.g., a bag or box) to thecustomer packaging loading station 118 that is proximate to thecarousel. There, individual product items are loaded from the tray intothe product packaging, and transported via the customer conveyor networkto a shipping dock 120 and thence to the customer in packaged form.

FIG. 1B is a simplified plan view of the embodiment of FIG. 1. FIG. 1Bshows an incoming pallet 122 of individual product items 124 (e.g.,apples) in bulk form. Each individual product item is inspected 125 andthen disposed in a known location on a tray 126. As shown at 128, thetray will be moved by the tray conveyor network to the appropriatecarousel and be placed into storage with the product items disposedthereon.

FIG. 1C is a simplified enlarged isometric view of the embodiment ofFIG. 1 illustrating six carousels 112. The enlarged view of FIG. 1C alsoshows an aisle conveyor 119 linking successive carousels, as isdiscussed further below.

FIG. 1D is another simplified plan view including a cut-away, of theembodiment of FIG. 1B. When an item from the tray is selected by acustomer, the tray will exit the carousel. In this particular view thetray of FIG. 1D is shown exiting a different carousel than in FIG. 1B,to which it may have been transported during an intermediate stage viathe tray conveyor network.

The unloading equipment 130 will remove from the tray, the particularitem that has been requested by the customer. The tray will return backinto the carousel until called again or empty.

Once the order is fulfilled, the customer conveyor network willtransport 131 the selected product item 124 together with the customerpackaging 131 to the shipping or distribution area.

It is noted that is some embodiments, the conveyor belts transportingthe customer package, may also move a passive or active device adjacentto some or all of the packages. That device may assist the placement ofthe items into the bag or box.

An example of a passive device could have spring loaded ‘landing pads’or an active height controlled ‘scoop’. With the latter, the items landinto the scoop (which can change height, and then drop into a package).Another example of an active device could be a robot. The devices (whichmay also have cameras) can be powered through the conveyor belt (wiredor induction) or be battery powered.

While the above description has indicated the loading of incomingproduct items onto trays, this is not required by all embodiments.According to alternative embodiments, items incoming to the producthandling system may arrive already disposed onto a tray.

An example could arise for standard items (e.g., boxes of cereal).There, each cereal box product item could be associated with data suchas an expiration date (and potentially an individual serial number).

Trays could arrive pre-loaded with non-standard items (e.g., produce,individually cut deli/cheese/meat . . . , others), and already beassociated with individual product item data such as images or othersensor data. The incoming tray may also include a mechanism formeasuring and storing environmental conditions since packing into thetray took place.

Such post-tray packing environmental conditions can include temperatureand vibrations/impact (e.g., via a G meter). Thus a system according toan embodiment would receive the pre-packed tray for handling, as well asdata associated with that particular tray.

According to certain embodiments, product handling systems may also beused to break down packages comprising one product each, into otherpackages having specific mixtures of product items. In such a ‘breakpack’ implementation, a distribution center breaks incoming packages ofone item type (e.g., typically from the factory) into mixed packagesintended for the neighborhood grocery store to replenish what has beenconsumed (e.g., 3 boxes of cereal A, 6 tuna cans, 3 salt shakers, etc.)

It is further noted that temporal factors (e.g., delivery urgency) mayimpact the manner in which individual product items are dispensed fromthe carousel into packaging. Thus where necessitated by a deliverydeadline, under a ‘split bag’ mode of operation a product handlingsystem could utilize more than one customer package to fulfill acustomer order. There, the more urgently needed item(s) would dispensedinto packaging first, followed by less urgent items being dispensed andshipped in a different package.

Product handling systems according to embodiments could alsopre-calculate and store estimates regarding time of order fulfillment.Such expected fulfillment data could be referenced by the system indeciding whether or not to resort to the split-bag mode in order to meetan urgent order.

An exemplary sequence of actions that may be performed by a producthandling system according to an embodiment, is now described inconnection with the flow diagram 1000 of FIG. 10. First, at 1002 theincoming material in bulk form is received for transfer into the traysif necessary.

The transfer of product material into the trays can be automatic orperformed manually. In the case of automatic transfer into dedicatedtrays, products are placed or dumped by a human or mechanical tool, orpicked by a robot onto a conveyor.

At 1004, the conveyor will sort the individual product items in anordered fashion for inspection. During an inspection process 1005, theindividual product items are imaged/inspected, and the images/data areassociated with each item (e.g., as metadata).

After inspection, the system will load 1006 each product onto trays andtransport the trays to the storage area (e.g., carousel). In some cases,the items are sorted in the Trays based on certain criteria (e.g.different size, quality or ripeness in different Columns)

Once transferred to a known location on the tray, at 1008 thatinformation will be maintained at least until the item has been placedin packaging for delivery. All or some of the data and metadata may bestored for longer to permit learning once feedback from a customer orother entity is obtained on the items.

At 1010, the Trays are loaded into carousels. The trays may be loadedmanually or automatically. The trays may be loaded through the same doorfrom the product items are later dispensed, or (as in the illustratedexamples) through a separate door or level. In one possible alternative,the trays remain within the carousels and the items are loadedindividually. Alternatively, the trays are loaded onto other trays nextto a carousel and placed in the carousel manually.

At 1012, when an item is selected by a customer and needs to bedispensed into packaging, the carousel may bring the tray to thedispensing door. There, the tray may be moved to the staging positionfrom which individual items will be dispensed. The tray may be moved bya conveying element within the carousel, or by an external liftingmechanism (e.g., belt or robot). Alternatively, the individual productitems could be dispensed from the tray while the tray remains within theCarousel.

From the staging position, at 1014 items are dispensed from the traysinto packages (e.g., delivery, inner, or transient) which are movingalong an aisle conveyor. Alternatively, the packages may be moved by anindependent motion vehicle/robot.

The aisle conveyor is positioned to support one or more parallel rows ofcarousels along its direction of travel. The aisle conveyor carries thevarious packages and potentially the inbound trays (e.g., with productitems) and outbound trays (empty, expired, or otherwise pulled out ofthe system).

The packages may be moving continuously or indexing and stopping nearthe staging positions. The packages may stop at every staging positionalong their aisle or as needed. The packages may step at fixed intervalsor as needed.

Packages may be placed into an aisle conveyor in coordination withplaced orders and the expected availability of items for conveying fromstaging positions along the aisle.

Alternatively, transient packages are places at fixed intervals.

Specific packages (e.g., including specific inserts if needed) may beplaced in the aisle conveyor for specific orders. If the exact number ofpackages needed per order is not accurately known in advance (e.g. acertain order may not fit in one package) than extra ‘buffer’ packagescan be placed in the aisle conveyor every several packages.

A package may travel on more than one aisle conveyor to collect theneeded items for a particular order. Alternatively, the delivery packagemay travel on one or more aisles and either/or transient packages orinternal packages will travel down one or more other aisles.

In this case the items may be merged into one or more delivery packagesmanually or automatically. Additional items deemed too large or fragilefor automation could also be manually added to the shipment at the endof the aisle conveyor.

As described herein, robots may be used in one or more stages of theproduct handling sequence. Examples of product handling activities thatcan be performed by a robot can include but are not limited to:

manipulating individual product items for inspection/imaging;

loading individual product items from bulk form onto known locations ontrays;

moving trays into/out of carousels;

dispensing individual product items from known tray locations intopackaging.

There can be one or more robots per location in the product handlingsystem.

Robots could be optimized in terms of weight and grip to the specificproduct items and/or system components (e.g., trays). One robot can haveseveral grippers, switch grippers, or each of the robots can have adifferent gripper.

The robots may reference various pieces of information to grab a productitem. Examples of such information may pertain to tray layout, imagestaken on the main or side-conveyor before, and/or dedicated images takenproximate to the robot (or on the robot's arm).

Robots can be of various types, including but not limited to cartesian,Selective Compliance Assembly Robot Arm (SCARA), cylindrical, delta,polar or a 4- or 6-axis articulated robot. Robot grippers can usepressure by rigid or flexible fingers, vacuum/suction, magnetic,electrostatic lifting, leaky vacuum (e.g., Bernoulli lifters), or acombination thereof.

A robot may use features within the trays in order to allow the liftingof product items, especially delicate/damageable items. In certainembodiments these can be rake/dustbin like, grabbing items which havebeen pin-lifted (e.g., from below with a potential balancing/locking‘thumb from above).

Packaging can pause adjacent to carousels or can have a parking area forone or more (e.g., three) customer packages. Multiple packages outsideof a carousel may all be reached by the robot, or they may need to moveto a specific parking position for the robot to reach.

Owing to its intimate relationship with the product being handled, thetray component forms one component of a product handling system.According to certain embodiments a tray may be wider than it is tall.

In some embodiments, a tray may comprise a formed sheet of plastic ormetal, that holds individual items or containers of items, in acontrolled order while the tray moves through the system.

Product items may be disposed on the tray loose (e.g., an individualapple). Alternatively, product items may be contained within a box(e.g., a box of tomatoes).

A tray may receive one type of item as a standard, but may also receivemore than one variety. This may occur for a low consumption items. Atray may receive a subcategory of an item (e.g., oranges between 3 and 4Ounce Vs oranges 4 and above.)

Some product items may be partially packaged. Examples include parsleytied with a rubber band, bunches of bananas, etc.

Product items may be individually labeled, for example with bar codesand/or RFID tags. The tray itself could have a bar code, RFID tag, orsome other marking to permit tracking if needed.

Product items within the trays may be arranged in rows and columns. Rowsmay be oriented parallel to the edge of the tray from which they areunloaded. Columns may be separated by barriers to allow product items toarrange within the columns utilizing grooves or other shaped features.

Specific embodiments may sort the items such that each bin is within adifferent Column. Here, bin may represent the quality, size, appearance,or a different product type.

Trays may be open, or have an opening wall at one or two of the edgesparallel to the rows. Alternatively, trays may be closed on all sides.

Trays may have openings, grooves, holes, or other features to allowlifting and/or movement of the product items from below. FIG. 2 shows aperspective view of a tray according to an embodiment. FIGS. 2A-2B showedge and plan views, respectively, of a tray embodiment. The tray 200includes grooves 202 and slots 204.

Certain tray features may hold a particular type of product or a certainvariety (e.g., stone-fruit of a given size range, oval shape,loose-leaf, etc.) FIG. 3 is a plan view of an embodiment of a trayhaving groove features configured to hold stone-fruit 300 items.

Trays exhibiting different features may be employed to effectivelyhandle various product types. Trays may maintain product items at aknown location that will not change during transportation.

Trays may be designed with sufficient spacing and other features topermit tasks such as imaging, scanning, sensing, and/or lifting. Traysmay exhibit features to allow removing the tray for packaging, orremoving the tray from the product handing system.

Embodiments of product handling systems may inspect the trays forcleanliness and integrity. Trays can be single or multi-use.

Some tray embodiments may feature a multi-use part covered by asingle-use layer. Such a single-use layer may comprise paper, plastic,cardboard, or other materials.

Embodiments of product handling system may include mechanisms forcleaning the trays after some or all the product items have beenremoved. Cleaning can include washing, brushing, electrostaticdischarge, UV, steaming, or other disinfecting techniques.

For purposes of imaging, product items that are to be transferred totrays, may be loaded into transfer conveyor belts. Such conveyor beltsmay have imaging and/or other sensing stations.

Examples of imaging techniques that may be employed can include but arenot limited to:

multispectral imaging

hyperspectral imaging

acoustic or acousto-optic sensing

optical spectrometers

3-dimensional imaging

UV imaging

visible imaging

infra-red (IR) imaging

mass spectrometry

x-ray imaging.

Examples of sensing technology that involve other than electro-magneticimaging can include but are not limited to:

chemical sensing (e.g., smell sensing technology); and

physical sensing (e.g., spring loaded firmness gauges or weighing—ofeither individual items or trays).

All items, or a sampling thereof may be imaged and/or sensed. Theproduct items can be imaged or sensed individually, or collectively orin sub-groups.

Weight can be estimated from produce size and/or from produce sizerelative to other produce in the tray when the total weight of the itemsin the tray is known.

Weight can be determined if the item is lifted by a robot. That weightcan be used to improve the estimation.

The imaging and/or sensing may occur while the product items are locatedon a transfer conveyor belt, are grabbed by a robotic arm, or as theproduct items are rolling or dropping before being disposed on a tray.

A transfer conveyor may be optically transparent to permit imaging ofproduct items from multiple sides and/or accessible angles. However,items can also be grabbed and lifted for inspection, imaging, orsensing, or simply raised such as via cushioned pins projecting thoughholes in the conveyor belt and/or tray.

Additionally, secondary imaging/inspection can occur during storage, orprior to dispensing a product into a package. Such imaging/inspectioncan be used as a final go-no go verification step after the attributesof the product item have been determined from previous imaging.

Other sensors and/or imagers may be used to confirm the successfultransfer of trays, items, and packages.

As described below in detail below, the product handing system maycomprise the trays utilized in conjunction with carousel elements. Oneor more sensors or cameras may be installed within the carousel.

Such carousel sensors/cameras could image some or all of the items asthe carousel rotates (e.g., a part of regular motion or to specificallyallow imaging). Several images could be taken during the motion topermit viewing from different perspectives, and even optionally 3-Dreconstruction of the product.

Some embodiments could install cameras at the top of the carousel toafford a view of the top tray. The cameras can be positioned on top inthe front and in the rear in order to ‘triangulate’ an image.

Items can be imaged once upon entering, periodically (such as every day,using the off-hours for example) or based upon other rules. Items can beimaged from the top while supported in the trays, and then flipped overindividually or as a whole tray to be imaged from the other side, orotherwise manipulated to rotate so they can be imaged from the otherside.

The imaging can take place on the main conveyor belt, and/or on the sideconveyors, and/or at the storage cabinets, and/or at the parking spotsbefore a robot manipulates the item.

A block may represent a plurality of trays. A given block may contain aset of items (e.g., all fruit and vegetables, all dairy, etc.) and isexpected to hold many (e.g., tens, hundreds, or even thousands) ofunique items.

A block may contain all or some of the items for sale. The block mayhold several trays of frequently accessed product items in order toensure rapid supply. Conversely, a block may have partial trays or evennone of rarely-accessed items that will instead be loaded upon demand.

The conveyor belt may lead to a set of modules that may be situatedorthogonally on one or both sides of the conveyor belt. The modules maybe dedicated for a specific variety of items (like fruit versusvegetables, packaged product items versus loose product items, etc.).Specific trays can be moved to side conveyors of individual modules asthey pass by on the conveyor.

According to embodiments, a loading mechanism may be employed to moveproduct items to and/or from the trays. In particular, once productitems have been inspected (e.g., by imaging and/or sensing), they aresorted onto trays.

One method to accomplish this sorting is to move the items alongconveyor belts. The trays (e.g., with slots) can be positioned above alifting conveyor. The lifting conveyor will lift through the tray andthereby allow the product items to be conveyed over the tray.

Once in place, the conveyors will move down and the items will rest onthe tray.

This is depicted graphically in the plan view of FIG. 4, where theproduct item 400 is disposed on the tray by the conveyor 402. The traycan then move on to storage (e.g., in a carousel).

Various stations could be used to match the variety of the items.Alternatively, an automatically adjustable conveyor could accommodateall or many of the possible varieties of product items.

In certain embodiments, trays having product items already disposedtherein, will be loaded into carousel elements. According to analternative approach, product items may be loaded into the trays thatare already in the carousel.

FIGS. 5A-D show different views of one example of a carousel 500according to an embodiment.

The carousels may be oriented in a horizontal direction, containingseveral tens of trays. The carousels may exhibit the ability to bringindividual trays to the transfer station 502, from which they can bemoved onto the staging position 504 (FIG. 5C).

The carousels may operate in a continuously indexing manner, bringingfollowing trays to an offload station. Alternatively, the carousel maycall trays in turn as needed.

Each carousel may operate with its own specific controlled environmentalconditions. Examples of such conditions can include but are not limitedto:

temperature,

humidity,

specific gases (e.g., ozone)

illumination

product-to-product proximity and interactions

mold growth.

According to embodiments, a carousel may have an opening (e.g., slit)for loading and unloading. The opening could be parentally open, anactively actuated gate, opened/closed by the motion of the tray, orclosed with a material permitting tray motion (e.g., a plastic sheet).

Carousels may be designed to allow placement of trays with differentspacings. This can permit handling of product items exhibiting differentsizes (e.g., heights).

Alternatively, trays may be designed with fixed spacing. A controlprogram may dictate which slot is appropriate for each tray height. Asingle carousel (one motor) could turn one or more columns of Trays.

Carousels may be designed with belt in cameras and sensors to continueto monitor the products as they age. One specific design could have thecameras or other sensors located at the top of the carousel path,inspecting the products in the tray passes past the top.

Additionally, there could be reject mechanisms built into the carouselsto allow items to be automatically removed from the system. One designcould employ lifting conveyors at the bottom of the carousel path. Theseconveyors are able to lift up and remove an item from the tray asneeded.

Trays can be removed from the carousel manually or automatically. Inautomatic embodiments, the tray can be transported out by a walkingbeam, a walking beam conveyor, or a conveyor or rollers built into thecarousel itself.

For walking beam embodiments, the carousel could stop with theappropriate tray in front of the dispense door. One or more beams wouldmove into the carousel, lift the tray up and pull it out of the door anddown onto a conveyor belt or rollers.

For the case where the conveyor is built into the carousel, each row ofthe carousel could have its own set of belts or rollers actuated toremove the tray. Each row could have its own motor, or a single motormay engage with the conveyors at the dispensing door.

Once located at a staging position, the tray will be in proximity to theaisle conveyor. This will allow items to be transferred from the tray atthe staging position to the packages.

Referring now to FIG. 6, conveyor belts 602 can rise up through openings604 in the tray 606. Product items 608 may be moved along a conveyorinto a position 610 from which they will be transferred 611 to thepackage, or into a separate transfer mechanism that will move theproduct items to the package.

FIG. 6 shows a perspective view of a tray with product items disposedtherein, interacting with a transfer mechanism 600. FIGS. 6A-B show planand edge views, respectively.

The product items may also be pushed, lifted, or dragged from the trayto that position for packaging. Product items could also be individuallypicked by a robot and an End of Arm Attachment (EOAT) specially designedfor secure engagement.

Once the product item or items have been removed from the tray, the traywill be conveyed back into the Carousel to store the remaining items.

Details regarding a transfer mechanism for moving the product itemsapart from the tray according to embodiments, are now discussed.Specifically, once an item is out of a tray, it can be moved to thepackaging by multiple methods.

One method is to have a robot pick up the item and place it into thepackaging. Here, there may be specific EOATs for various classes andsizes of products. Some EOATs may have suction cups, others may havegrippers, still others may shovel items up.

An alternative method is to have the product items fall off of the endof a conveyor into a receptacle such as a rigid box or flexible bag,which may or may not have padding to break the fall. FIG. 9 shows anexample of such an embodiment featuring receptacle 900.

The item could fall onto a spring-loaded slide or flap that drops downinto the bag or box to soften the landing. The bag or box could beactuated to lift the box up to the edge of the conveyor. The box couldalso be constructed in a manner that the one more sides of the bag orbox temporarily collapse to allow items to be brought into them.

A third possible approach is to employ a vertical conveyor whose belthas large flaps to hold the product item. FIG. 8 shows a side view ofsuch an embodiment which includes flaps 800.

The conveyor would lower into the bag or box, or the bag or box couldrise up around the conveyor. The conveyor could be actuated in a mannerto select the location to drop the item in the box or bag.

A fourth approach is to have the item move above the package on aconveyor. Once in position, the conveyor pivots down like a trap door.FIGS. 7A-C are side views showing the operation of one such embodiment.

The conveyors could lower down into the bag, controlling the decent ofthe item until it is beyond the reach of the conveyor. The belts couldbe made of a thick foam that helps capture the items on the way down.There could be two or more belts on each side with various spacing thatcan process various sized items. Each station could be designed for acertain shape, weight or volume. Or, one design could be actuated in away that the location of each belt is movable to handle item variety.

A fifth method is to push or pull the product items from the back of thecolumn or row of the tray. This could be accomplished from above via amechanical gantry system, from below via an actuator through a slot, orutilizing a robotic arm.

A sixth possible method is to convey the product items through a holewith a stack of mesh or plastic rolled onto it. As the items are pushedthrough, the bag will roll forward until full and then be terminated andclosed.

Certain embodiments could feature one transfer mechanism per tray. Otherembodiments could feature one transfer mechanism per column or row ofthe tray.

There could be one or more transfer mechanisms per carousel. Thetransfer mechanism may be actuated in such a manner as to move betweenpositions. Alternatively, the individual product items could be liftedonto conveyors at the staging position and then conveyed to one or morefixed transfer mechanisms.

Embodiments of product handling systems may permit customer selection ofspecific items. In some cases, e.g., produce or other non-uniformproducts (meat, deli cheese, etc.,) the consumer may be offered theability to choose from a number of specific items.

According to such embodiments, images or other (e.g., sensing) dataprovided to customer, may allow him or her to successfully execute theproduct selection. The specific item may be reserved for a certainamount of time allowing the selection. Once selected by the customer,the specific item will be vended from the carousel.

Certain embodiments may allow dispensing only from the front row (e.g.,closest to the package). There, customer selection may be limited to thefront row, and/or selection of sequential items in the same column willbe possible.

In alternative embodiments, a series of conveyors could move some itemsout of a column or row, so that an interior item can be vended to thecustomer. Then, the conveyors could move the remaining items back intothe tray for return to storage.

Embodiments could offer the customer multiple qualities for evaluatingwhen selecting their item. The customer could evaluate color pictures ofthe item from multiple angles. The customer could also be offeredhyperspectral images.

A system according to embodiments could show a customer a rating thatsignifies the firmness of the item. The rating could be on a scaleshowing maximum firmness for similar items. For example, the firmestpear (right off a tree) could be rated with a value of 10, while a pearon the day of expiration could be rated a value of 0.

The scale could be adjusted based on the season of the item. Forexample, off-season produce might have a narrower range of possibleratings.

Similarly, a rating scale could be shown for various characteristics ofproduct items. Examples can include but are not limited to:

sugar content in fruit

the ripeness of certain items like bananas or avocados

an overall rating that shows our compiled assessment of the overallquality of the item.

The rating scale could be any set of numbers or an A, B, C, D, F scaleor similar.

Product handling systems according to embodiments may include mergingand/or shipping areas. At the end of the aisle conveyors, other conveyorsystems (or product moving mechanisms) may transfer packages either toother aisles or to a shipping area.

Multiple packages can be merged into one or more delivery packages. Forembodiments featuring transient packages, individual items may be liftedor otherwise transferred (such as poured) into delivery packages.

Product handling systems may feature a buffer station where packages arewaiting. This waiting may be for other Packages in the same shipment, orfor other reasons.

The nature of delivery packages according to embodiments, are nowdescribed. Delivery packages can be bags, simple boxes, or boxes havingspacers inside.

One packaging option is for single layer stackable boxes. Another optionis to have boxes with several heights, with the box ultimately useddetermined by the largest item it is designated to hold.

A box height could be a designed to have 1 or more layer, with apackaging material for contact with fragile items, and remaining layersfor regular items. Packaging for a fragile layer could have a formedplastic or cardboard insert sized to fit various types of items (e.g.,round, oblong, flat, etc.)

Packaging may be single use from recyclable material, and may berecycled. Packaging could also be designed to be returnable from thecustomer.

Transient packages can be individual bags or boxes with items collectedinto a larger box. Transient packages can be a large tray on which itemsin an order are placed for transport to the packing area.

Several delivery packages can be collected for one customer order. Thiscan, for example, be by employing vertical stacking or nylon wrapping orother technique.

Embodiments may employ techniques for controlling the queuing and/ortiming of product movement through the product handling system.Particular embodiments may use information from orders received, as wellas from orders that are expected. The latter may be accomplished throughmachine learning techniques that are keyed off of past behavior.

One example of product movement that may be coordinated, is to sendpackages into aisle conveyors. This package sending may be for delivery,internal movement only, or for transient packages.

Another example of product movement that may be coordinated according toembodiments, is to bring trays to an unload position from the carouselsand to the staging position. This movement may need to be done in timeto preempt the loading of specific items into the packages.

Still another example of product movement that may be coordinated isaction to bring trays, transient packages, or customer pack boxes to therobot loading area. The activity of the conveyors, carousels, andloading mechanisms may be coordinated in order to maximize systemthroughput and minimize time from receipt of the order to packaging.

Embodiments of product handing systems may function to optimize locationof items on trays, and the location of trays in carousels. This allowseach carousel and packaging location to be fully utilized.

Embodiments may track performance of the various components. An overallsystem health may be displayed to the users.

Embodiments of product handling systems may also operate to plan theoptimal packing, for example calling upon specific pack boxes as neededto fulfill each order. In one embodiment, the system may place larger,heavier items at the beginning of a cue so that those items are at thebottom of the packaging. Then, as the packaging gets filled,progressively lighter and more fragile items can be positioned on thetop.

Embodiments could also function to determine where in the box aparticular item should be placed. This allows knowing where productitems are located through the entire packaging sequence.

The imaging and sensing data collected from the product items (as wellas other available information such as images or othermeasurements/information taken before arrival at the facility) may beused to attribute properties to the item. Such other information caninclude data provided on barcodes or similar mechanisms such as RFIDtags. These can be located on the individual item, on the incoming tray,or be traced to the incoming shipment. In some embodiments the otherinformation may be measured ‘off-line’, such as a sample of fruits thatare sent to lab tests or tasted.

The customer can be offered various packages if desired. In one example,produce could be selected based upon an expected day of ripeness. Thusan instruction received from the customer may be to “select 7 bananas,one that ripens every day for the next week”. Alternatively, a customercould request a box of fruit per day, each box ripening as needed.

In another example, a customer could select kits specifically intendedfor a certain recipe. Thus bananas slated for use in banana bread, couldbe ordered as bananas slightly paste their ripe stage.

Further alternatively, a customer could order a lasagna kit. During theselection process, the customer could be offered a list and choose someor all of the ingredients from that list.

Embodiments may allow external links to be placed on recipe sites, sothat a user can add the recipe to their list of saved recipes. Forexample a customer may prefer the pasta sauce available from aparticular website, and click the save recipe option. Then, that productitem will be added to the customer's profile.

Later, the customer might look through their recipes and plan out mealsfor the week by selecting each meal and the day it is expected to beeaten. Embodiments could then sort through the order and select items ofthe best quality on the day scheduled for consumption.

According to embodiments, the product handling system could also belinked to other Internet-of-Things (IoT) devices that are able todetermine or predict particular items already present in a customer'srefrigerator or cupboard. Those particular items may then be able to beautomatically ordered on a recurring time period. Alternatively, when arecipe is ordered, items already possessed by the customer can beautomatically removed from the list.

A system according to embodiments could be linked to voice controlleddevices.

This allows a user to add an item to their grocery list so that it canbe saved for a future order. A customer could accomplish this by askingto have items saved to their cart, or by notifying the system of adeficit of certain item. Information relating to particular dishes orrecipes could be saved in a similar manner.

System users may have the ability to save past quality preferences. Thusif a customer ordered B level zucchini (for example), that customercould mark a box saving that preference for all future zucchinipurchases. This shortens check out time and improves consumerconfidence.

Imaging and other (e.g., sensing) parameters may be stored forsufficient time to allow comparison after customer feedback has beenreceived. (This may be about a week for fresh produce purchases.) Thestored information could be the raw data or a subset thereof that hasbeen processed (e.g., for compression or encryption).

Parameters or ratings can be referenced to offer different pricingschemes or dynamic (adjustable) pricing based upon quality and/orsupply-demand variations.

A customer can provide feedback on particular parameters of thedelivered product. Examples of such parameters can include but are notlimited to the quality, ripeness date, blemishes, or other parameters ofpurchased produce.

Systems according to embodiments may utilize machine learning to processfeedback information together with other collected parameters (e.g.,images of product items). This allows the system to improve an accuracyof predicting general properties (e.g., ripeness day, color, farm, andother attributes) as well as the preferences of a specific customer.Such data processing can aid customers in obtaining preferences, allowsellers to evaluate supplier performance, and permit suppliers tocollect valuable quality control information and improvement feedback.

The user interface (UI) may include virtual or augmented reality imagesor scenarios. In one example, a user could enter a virtual supermarketwhere they can inspect and select the specific to be placed into theircart. Such a virtual store environment can be optimized to minimize timespent by down selecting items in an order they are shown according topreferences.

Product items not normally bought by a particular customer can bepresented for inspection (including physical handing).

Recipes offered by the system accompanying product items, could beavailable for customer review to inform about possible uses for theproduct. If a recipe is selected, the user can be solicited to add theother recipe items to their cart. Alternatively or in addition, a userselecting a recipe could enter a virtual environment displaying finisheddishes or recipes for ordering other ingredients as desired.

According to embodiments, a product handling system can provideestimated packing time based upon factors including but not limited to:

the order contents

other orders in the queue, and

the known content of the block containing other orders.

A product handling system according to embodiments could suggest changesin the order to reduce pack readiness time. The system could alsoactively suggest changes to the order based upon price considerations,for example to meet a budget requirement.

Example

An example of a product handling system is now described. An embodimentof an automated system allows rapidly distributing a large number (onthe order of many 10,000's) of unique incoming products into packagesfor customer delivery.

According to one embodiment, the system provides an automated grocerystore offering rapid (e.g., minutes) fulfillment and packaging. Inparticular, grocery items typically involve 1,000s or 10,000s of uniqueitems.

For example, individual product items in the field of fresh produce, maycomprise a large volume of associated data. That is, an individual fruitor vegetable (e.g., an apple) may be associated with one or more of thefollowing pieces of information:

product item identifier (ID)

size

color

variety

harvest date

source (e.g., farm)

visual inspection result

non-visual inspection result (e.g., softness, gas sample, many others)

This information could be stored in electronic format in anon-transitory computer readable storage medium. In one example, theinformation could be stored as fields in a data object stored in adatabase. The information could travel associated the particular productitem, together with product handling specific data (such as a trayidentifier, tray location identifier).

Upon user selection of the particular product item (e.g., apple), theproduct handling system could dispense the item to the consumer togetherwith packaging (e.g., box, bag, together with any insert) within a shorttime frame.

While this particular example describes an embodiment that is designedfor use in the packaging and delivery of fresh produce, this is notrequired. Other potential applications may call for the rapid fulfilmentfrom a large number of product options, for example in the area ofrecycling of clothing or other items.

Particular embodiments may be especially suited to allow rapiddistribution of a large number (on the order of many 10,000's) of uniqueincoming products into packages for customer delivery. By contrast, aconventional on-line retailer may carry millions of different individualitems in warehouses for delivery.

Embodiments may thus offer a space efficient solution that may be localwithin proximity of customer demand. However, some embodiments could belocated in larger warehouses outside of densely populated areas. Systemsmay permit a high level of quality control, allowing customers to avoidthe inefficient travel to and from a store in order to select individualitems for freshness and relevance.

In summary, embodiments may offer optical (hyperspectral, potentially3D, potentially multi-angle) or other (like acoustic, pressure gauge,gas sensing systems, spectroscopy) inspection of some or all incomingmaterial (e.g., loose produce) in bulk form.

Embodiments permit customer selection of specific product items (e.g., aparticular tomato) based upon one or more of the following:

specific information (images)

specific or aggregated sensor data

typical images (of others that we sorted into same bin)

meta data accompanying the items (manufacturer's location and pick dateif applicable, storage and transportation conditions, etc.)

the rank of a specific item within a larger aggregate of items

other criteria (such as size, weight, volume, color).

Embodiments may offer storage of product items within carousels thatcontrol multiple environmental factors (e.g., temperature, humidity,product to product proximity and interactions, mold growth, gasdetection). Such a carousel-based storage system could be designed toallow rapid packing of individual products to provide rapid orderfulfillment.

Embodiments may allow the vending of fresh produce into either adelivery package, an inner package, or a transient package. Here, aninner package represents a tray that will later go into a box withothers. A transient package serves just to carry the product item(s) toa place where a shipment package is consolidated.

In the case of direct vending into a delivery package, that packagingmay include a generic form of spacing or cushioning or an insert thatwill be placed before loading based on the expected items to be placesinto that delivery package (e.g., round indents for the packaging ofstone fruit).

According to certain optional embodiments, inner packages, and/orindividual items, and/or transient packages may be merged into adelivery package if appropriate.

In some cases, other material may be added to prevent damage to items inshipping or packaging. Examples can include but are not limited tocardboard, shredded paper, inflatable plastic bags, or other fillers.Alternatively, the individual dispensed product items may be coveredwith another flat or shaped surface or a stretchable or inflatablemembrane for protection during shipping.

Embodiments may permit individual item selection by a consumer based onimaging and other sensor data. This selection may be enhanced bylearning consumer preference(s) based upon image, collected data, andother sensors. This may be achieved independently, or in connection withautomated features.

Various product handling systems according to embodiments may exhibitone or more of the following features. Certain embodiments may hold fora customer, a specific selected unique product items while they are intheir cart. Some embodiments may save quality preferences of a customerfor future orders.

Particular embodiments may feature recipe buttons that allow user tosave a recipe, and select one or more particular product items of thatrecipe as part of an order. Product bundling features may facilitate auser ordering whole recipes, kits or a series of items. Variousembodiments may allow optional browsing and selection by meal optionsinstead of individual items.

Voice controlled additions to a cart may be allowed for future orders.Various embodiments may implement dynamic pricing based upon factorsthat can include but are not limited to expiration dates, consumerfeedback, and conveyor load. Some embodiments may dispense product itemsfor packaging according to expected expiration date.

FIG. 11 is a simplified diagram illustrating an embodiment of a producthandling system 1100 according to an example. Specifically, a user 1102interacts with a processor 1104 via a graphical user interface (GUI)1106.

The processor is in communication 1107 with both a database 1108 in anon-transitory computer readable storage medium 1110, and the variousother system components 1112. Specifically, those other components serveto sort 1114 incoming items, with the processor assigning each a productitem identifier that is stored in the database.

The individual product items are then inspected 1116. The processorassigns an identifier 1118 to each of the inspections, the results ofwhich are stored in the database together with the product identifier1120 and the inspection result identifier 1122.

As described extensively above, the individual unique product items arethen loaded 1124 into tray(s). Again, the processor stores in thedatabase, the corresponding tray identifier 1126 and tray locationidentifier 1128 associated with each individual product item identifier.

Next, the trays are moved to the carousel for storage 1130. A carouselidentifier 1132, and a location 1134 of the tray within the carousel,may be stored by the processor in the database associated with the otherIDs, thereby allowing tracking of tray and product item.

The specific data relevant to various product items is then communicated1136 from the processor to the GUI, where it is displayed 1138 to theuser. Based upon this displayed product item data, the user provides aninput to select 1140 a particular product item.

The processor receives this selection, and in response communicates aninstruction 1142 to the carousel. Based on that instruction, thecarousel dispenses 1144 the unique product item into packaging. Finally,the product item together with the packaging are delivered 1146 to thecustomer.

Clause 1A. A method comprising:

sorting a first unique product item from a plurality of product items;

assigning a first identifier to the first unique product item;

conducting a first inspection of the first unique product item;

storing in a non-transitory computer readable storage medium, a firstinspection result associated with the first identifier;

disposing the first unique product item on a tray at a location that isassigned a second identifier;

storing in the non-transitory computer readable storage medium, thesecond identifier associated with the first identifier;

moving the tray to a carousel; and

storing the tray in the carousel under a controlled environmentalcondition.

Clause 2A. A method as in clause 1A further comprising:

communicating the first identifier and the first inspection result to acustomer;

receiving from the customer an instruction including the firstidentifier;

in response to the instruction, referencing the second identifier basedon the first identifier;

in response to the referencing, causing the carousel to dispense theproduct item from the first unique location into a packaging; and

distributing the product item in the packaging to a consumer.

Clause 3A. A method as in clause 2A wherein:

conducting the first inspection comprises conducting an opticalinspection; and

the first inspection result comprises an image of the unique product.

Clause 4A. A method as in clause 2A further comprising:

conducting a second inspection of the first unique product item;

storing in the non-transitory computer readable storage medium, a secondinspection result associated with the first identifier; and

communicating the second inspection result to the customer along withthe first identifier and the first inspection result.

Clause 5A. A method as in clause 4A wherein the second inspection isother than optical in nature.

Clause 6A. A method as in clause 1A wherein the controlled environmentalcondition is selected from temperature, humidity, illumination exposure,and proximity to another unique product item.

Clause 7A. A method comprising:

communicating to an interface, an inspection result for a unique productitem stored under a controlled environmental condition within acarousel;

receiving from the interface, a selection of the unique product item;

in response to the selection, referencing a data object stored in adatabase to correlate a first identifier of the unique product item witha second identifier of a specific location within a tray in which theunique product item is disposed; and based upon the second identifier,communicating an instruction to the carousel to dispense the uniqueproduct item into packaging.

Clause 8A. A method as in clause 7A wherein prior to the communicatingthe method further comprises:

sorting the first unique product item from a plurality of product items;

assigning the first identifier to the first unique product item;

conducting a first inspection of the first unique product item;

storing in a non-transitory computer readable storage medium, the dataobject associating the first identifier and the inspection result;

disposing the first unique product item on the tray at the specificlocation assigned to the second identifier;

storing in the non-transitory computer readable storage medium, the dataobject associating the second identifier with the first identifier;

moving the tray to the carousel; and

storing the tray in the carousel under the controlled environmentalcondition.

Clause 1B. An apparatus comprising:

a first carousel configured to receive from a first transport network, atray including a unique location associated with a first identifier andbearing a unique product item associated with a second identifier, thefirst carousel configured to maintain the unique product item under afirst controlled environmental condition;a transfer mechanism configured to dispense the unique product item intoa first packaging at a staging location; anda second transport network configured to move the unique product itemand the first packaging from the staging location for distribution to acustomer.

Clause 2B. An apparatus as in Clause 1B wherein the first packagingcomprises a delivery packaging.

Clause 3B. An apparatus as in Clause 1B wherein the second transportnetwork is configured to move the unique product item for distributionvia a second carousel.

Clause 4B. An apparatus as in Clause 3B wherein the first packagingcomprises a transient packaging.

Clause 5B. An apparatus as in Clause 3B wherein the second transportnetwork comprises an aisle conveyor belt.

Clause 6B. An apparatus as in Clause 1B wherein the transfer mechanismcomprises a robot.

Clause 7B. An apparatus as in Clause 1B wherein:

the tray defines an opening; and

the transfer mechanism comprises a member moveable into the opening tocontact the unique product item.

Clause 8B. An apparatus as in Clause 1B wherein:

the first transport network is in a first plane intersecting thecarousel;

the second transport network is a second plane also intersecting thecarousel; and

the carousel is configured to move the tray between the first plane andthe second plane.

Clause 9B. An apparatus as in Clause 1B wherein:

the first transport network is in communication with a first end of thecarousel;

the second transport network is in communication with a second end ofthe carousel opposite to the first end; and

the carousel is configured to move the tray between the first end andthe second end.

Clause 10B. An apparatus as in Clause 1B further comprising aninspection station configured to interrogate the unique product itemprior to being disposed to the unique location.

Clause 11B. An apparatus as in Clause 10B wherein the inspection stationincludes an optical camera.

Clause 12B. An apparatus as in Clause 1B wherein the first packagingcomprises a bag or a box.

Clause 13B. An apparatus as in Clause 1B wherein the first packagingfurther comprises an insert.

Clause 14B. An apparatus as in Clause 13B wherein the insert comprises afiller.

Clause 15B. An apparatus as in Clause 1B further comprising a loadingstation configured to dispose the unique product item in the uniquelocation of the tray.

Clause 16B. An apparatus as in Clause 11B wherein:

the tray defines an opening; and

the loading station comprises a member moveable into the opening tocontact the unique product item.

Clause 17B. An apparatus as in Clause 16B wherein the member comprises arobot.

Clause 18B. An apparatus as in Clause 11B wherein:

the tray defines an opening; and

the first transport network comprises a member moveable into the openingto contact the unique product item.

Clause 19B. An apparatus as in Clause 18B wherein the member comprises apin.

Clause 20B. An apparatus as in Clause 19B wherein the first transportnetwork comprises a conveyor.

FIG. 12 shows a perspective view of a product handling system accordingto an embodiment. Product handling system 1200 includes multiple banks1202 comprising six carousels 1204 each, that are oriented in two rowsback-to-back.

Storage of particular items within a specific carousel, is typicallydetermined at least in part by the conditions of that carousel. Certainitems may be stored in groups calling for the same or similarconditions.

For example a set of fruit product items that call for a suite ofconditions such as:

high relative humidity (RH),

low temperature,

low oxygen, and

high CO₂,

can all be stored together in the same carousel.

Product types calling for different storage conditions, may all bestored together in a different carousel. For example, in contrast withfruit product items, certain dairy product items may call for:

low temperature, and

normal RH conditions.

Each carousel may thus be characterized in part by its particular suiteof environmental conditions, as well as by its physical location withinthe system.

FIG. 13 shows a side view of one carousel bank of the system of FIG. 12,with the side wall of one carousel cut away. This system utilizes atwo-level transport path configuration, with trays 1206 arriving forloading on an upper level 1208, and trays being retrieved and productsbeing dispensed into bags 1210 on a lower level 1212. Details regardingthe product retrieval and unloading process for this particularembodiment, are further described below.

Returning to FIG. 12, it is noted that multiple carousels of the sametype (e.g., “Berries”) may be present in a single system. Thisredundancy may be introduced in order to distribute the items throughoutthe system and increase bag path efficiency. Unwanted bottlenecks in theflow of materials through the system can be avoided, e.g., whereproduct(s) of a particular carousel are popular and hence need to bedispensed/replenished, often.

It is further noted that the arrangement of carousels, and even banks ofsame, may be carefully determined in order to enhance the efficiency offlow of materials through the system. For example, carousels storingitems typically purchased together, may be grouped to be physically nearone another, thereby reducing inefficiencies of travel time/distance.

The specific location of product items in the carousels can be adjustedover time. This may be done in response to the identification of productpurchasing behaviors that emerge for particular customers or subsetsthereof. Such trends may be determined with the aid of intelligentlearning approaches that are trained on past customer behavior.

Other considerations may influence carousel location within the system.For example, as described herein the placement of items within a bag maybe carefully determined, with fragile products being dispensed last, ontop of more durable items. Thus, carousels storing such fragile productsmay be located downstream in the process flow, with items ordered in away that those slated for the bottom of the bag drop first, and thoseslated for the top of the bag drop last.

In addition to relative fragility, a dispensing order may also dependupon relative product item size. Certain approaches may seek to dispensea large item (e.g., a tall box of cereal) early in the sequence, andthen dispense smaller products around it.

Returning to FIG. 13, the cut-away view shows the storage of multipletrays of products within the rotating carousel. Depending upon factorssuch as the number and quantity of products to be stored, carousels maybe of varying sizes. Examples of standardized carousel heights caninclude but are not limited to (in feet): 13.5, 17.25, 21, 24.75, and28.5. Other heights may also be used.

Conditions within each carousel are carefully maintained, and theenvironment of each selectively modified in order to extend productlife. Environmental conditions that may be relevant for storing productsin a desired state, can include but are not limited to:

time of storage;

temperature;

relative humidity;

gas levels (e.g., O₂, CO₂);

ethylene removal;

light exposure;

cross contamination;

Volatile Organic Compound (VOC) levels;

mold spore levels; and

cross contamination.

Detection of such conditions can take place within the carousel itself,checking multiple trays at a same time. Alternatively, such conditionscan be detected on an item-by-item basis during product intake andsorting, or prior to dispensing.

Hence, products that are desired to be maintained under similarconditions, may be stored within the same carousel. Incorporated byreference herein for all purposes, is the following document:Agricultural Handbook No. 66, “The Commercial Storage of Fruits,Vegetables, and Florist and Nursery Stocks”, printed by the U.S.Department of Agriculture as revised February 2016. This publicationprovides a table listing compatible fresh fruits and vegetables during7-day storage.

Considerations other than environment may dictate the storage ofproducts within or outside of, the same carousel. For example, consumersafety reasons may require the segregation of product items that are thesource of food allergies. Hence, the certain carousels may be dedicatedto the storing of nuts only.

Details regarding the dispensing items from the tray with a conveyorfrom below, are now provided. In particular, the product retrieval anddispensing apparatus according to particular embodiments, may offerthree services:

removing tray from carousel;

removing items from tray; and

placing items from tray into a bag.

FIG. 14 shows a perspective view of bag 1210 (shown with handles foldeddown), that is approaching and halting in front of the carousel on thetransport path. At this time, the frame 1220 rises up to the level ofthe carousel opening 1222.

FIG. 15 shows a perspective view of forks from the frame extending 1224into the carousel and lifting a tray 1226 present therein. FIG. 16 showsa perspective view of the frame 1220, showing the forks 1230 extendedtherefrom. FIG. 17 shows an end view of the frame 1220, showing the fork1230 extending underneath the edge of the tray 1226 having product 1228disposed thereon.

Details regarding the architecture of the machinery along the retrieveaxis, are now described in connection with FIGS. 18-19D. In particular,FIG. 18 shows a perspective view of the frame prior to the extension ofthe forks.

FIGS. 19A-D show enlarged views of the front of the frame during theretrieval process. In FIG. 19A, a drive flipper 1232 is spring loaded inthe up position. FIGS. 19B-D show that when the fork end effector isextended by the rearward drive gear 1234, a roller interface 1236 thatrides on the drive flipper pushes it down into position.

After some amount of travel, the forward drive gear 1238 engages therack. The gearing is designed so that both drive gears can interact withthe rack at the same time and pass off the rack to each other. At fullextension only, the forward drive gear is engaged.

Between FIGS. 19C and 19D, the flipper raises the forks to cause thetray to come off of its supports in the carousel. A benefit is that theback side of the forks do not stick out into space, and can be flushwith the end of the tool.

FIG. 20 is an end view of the frame with the extended forks engaging thetray as part of the retrieval process. As the forks lift the tray fromthe carousel, product conveyors 1240 move laterally 1242 to the specificpick location (i.e., underneath the row of the tray from which items areto be dispensed).

FIG. 21 shows a perspective view of a product conveyor 1240 whichincludes a slide 1244. FIG. 22 shows a perspective view of a tray (hereempty for illustration) lowering over the product conveyors. FIG. 23shows a perspective view of the product conveyors engaging to lift theproducts from the lowered tray.

FIGS. 24A-B show side views of the product conveyor in non-extended andextended positions, respectively, including belt 1246 and fixed pulleys1248. FIG. 25 shows a side view of the extended product conveyor bearingproducts lifted from the tray. In particular, the product conveyor canslide forward off the tray and over the customer bag, in order to dropthe item in a specific location.

Here, the belt stays tensioned as it moves between fixed pulleys.Pulleys 1251 move forward and back. They stay between the outsidepulleys. This helps to ensure that tension is maintained in the belt.

Belt movement and sliding action may be driven by two splines 1250 andgears so that motors are fixed under the unit, making it easilyremovable and cleanable. As the conveyor, 1240 slides left and right tofit into the correct slot in the tray, the spline will stay engaged withthe gears that drive the sliding action and the belt movement.

FIG. 26 shows a front view of the item being moved off of the extendedproduct conveyor into the bag.

FIGS. 27A-C are side views showing a sequence of disposing a productinto a bag according to an embodiment. At FIG. 27A, the customer bag2700 starts bunched up at the top. FIGS. 27B-C show that as the bagbecomes filled, a motor lowers the platform 2702 that the bag is sittingon, and additional items 2704 are placed on top.

A benefit of this approach is that the drop height is always the sameheight. Additionally, some items can be positioned in specific locationsby controlling one or more parameters such as:

bag height;

conveyor position; and/or

bag location.

In an example, a product in the form of a bottle of wine could beinitially disposed in the tray, oriented parallel to the productconveyors. As the bottle tips over the edge of the product conveyor, thebottom of the bottle will reach the bag and stop at an angle, restingagainst the product conveyor. The product conveyor can then push the topof the bottle forward until it is sitting upright.

As described herein, one benefit of product handling systems accordingto embodiments, is that they may allow for the full diversity ofproducts of a conventional grocery store, to be simply organized,stored, and then accessed. One feature that is particularly helpful inthis regard is the tray element.

Particular embodiments may feature two different tray types, each typeconfigured in various sizes to receive different products. A first traytype is designed to store products that are round or cylindrical inshape, and hence exhibit a roughly circular cross-section

FIGS. 28 and 28A-B show overhead and cross-sectional views,respectively, of such a first tray type 2800 according to an embodiment.Here the tray is supporting products 2802 of cylindrical shape having aroughly circular cross-section. These trays can also hold sphericalitems.

By contrast, FIGS. 29 and 29A-B show overhead and cross-sectional views,respectively, of a second tray type 2900 according to one embodiment.Here the tray is supporting products 2902 of cuboid shape having aroughly rectangular cross-section. This tray type may differ from theother tray type in not having concave features configured to receiveproduct items having a rounded cross-section.

Given these two basic types, various sizes of row dimensions may allowfor the support of a wide variety of potential products, as is describedin the following tables.

TABLE Group I (circular product cross-section) Range of Product Tray #Cross-Sectional Diameter (mm) Example Product I.1 ~30 to ~47 LooseCarrot I.2 ~47 to ~65 Avocado I.3 ~55 to ~87 Orange I.4  ~80 to ~123Bell Pepper I.5 ~115 to ~155 Cauliflower Head I.6 ~140 to ~207 BananaBunch

TABLE Group II (rectangular product cross-section) Tray # Range ofProduct Width (mm) Example Product II.1 ~55 to ~75 Spaghetti Box II.2~75 to ~95 Cheese block II.3  ~95 to ~115 ½ Gallon Juice II.4 ~115 to~145 Bag of Flour II.5 ~145 to ~195 Gallon of Milk II.6 ~195 to ~300Frozen Pizza

As previously described, embodiments may employ sensing ofcharacteristics of products that are stored in the carousel, in order toensure delivery at predetermined levels of freshness. One way that thiscan be done is by product imaging.

Specifically, optical characteristics of specific products at specificstages of freshness can be taken and recorded, in order to provide onemethod of quality measurement. FIG. 30 shows a simplified view of suchan approach, where a series of optical images 3000 a-e have been takenof a product (here a bunch of bananas) at different stage of itsfreshness lifetime, ranging from as harvested, to pre-ripe (e.g. green),to ripe, and then to over-ripeness (e.g., suited for banana bread), andfinally to no longer consumable.

Various optical characteristics of the image may be measured for eachpicture and stored, and a database created from the measurements.Exemplary optical characteristics which can be sampled by imagingaccording to embodiments, can include one or more of the following colorproperties:

hue;

saturation; and/or

brightness.

Using machine learning techniques in combination with artificialintelligence (AI) approaches, these images taken of produce along itsfreshness lifecycle, can be automatically segregated into qualitygroupings, depending upon how their optical properties (e.g., colors)change over time.

In one example, thirty-eight (38) images were taken of the specificbunch of bananas over its freshness lifetime. Saturation color propertydata for pixels of each of these images was sorted into 16 bins. Then,over ten iterations of K-means clustering, the resulting five groupingsof product quality in FIG. 30 emerged.

FIG. 31 is a resulting histogram of the saturation data, showing afraction of pixels exhibiting different values. By referencing suchcolor properties stored in a database in connection with additionalimages that are acquired of new incoming produce, embodiments canimmediately and accurately assign a freshness category to that incomingproduce.

In summary: calculating one or more color properties of pixels invarious optical images of a perishable product (such as produce), andthen using that color property data in order to develop a database, canprovide a model of the typical freshness life cycle of that produce.Such approaches would allow future color measurements to accuratelypredict a current status of newly-received instances of that perishableproduct during its freshness lifecycle (e.g., from harvested, topre-ripe, to ripe, to over-ripe, to no longer useable).

It is noted that the freshness correlation just described above, is notlimited to being based upon the sensed optical property of saturation.Other pixel optical data (such as from hyperspectral or multi-spectralimaging) can be referenced in the database.

Moreover, freshness prediction need not be based exclusively upon sensedoptical characteristics. To enhance accuracy of the product freshnessmodel and database, other factors could also be correlated within thedatabase with the product and considered, including but not limited toone or more of:

diameter;

length/width/height (LWH);

weight;

shape;

blemish count;

blemish size;

detection of the presence of mold or insects;

elapsed time from harvesting;

firmness;

texture;

sugar content;

In addition to the inspection data mentioned above, data incoming withthe product item from the source, could also be included in the databaseand correlated with expected freshness and other AI procedures. Examplesof such incoming data types can include but are not limited to:

source (e.g., grower);

location (e.g., orchard #);

heat treatments;

radiation treatments;

pesticides;

organic certification;

Genetically Modified Organism (GMO);

days of sunshine;

days of rain;

harvest date;

days in storage;

transport company;

transport type;

transport conditions (e.g., temperature, duration); and

time processed.

One or more of the data types described above, may be further correlatedwith product item data that is also stored in the database. Examples ofsuch product item data can include:

unique product item identifier;

Stock Keeping Unit (SKU);

locations (including past locations) of the specific product item withinthe product handling system (e.g., by carousel, tray, tray row, rowposition);

textual descriptor (e.g., “golden delicious apple”);

product classification (e.g., according to taxonomies set forth byinternational/national regulatory bodies, such as Import & Export);

brand name;

varietal;

vintage;

special notices (e.g., potentially allergenic, citations tofederal/state/local regulation—e.g., Food & Drug—FDA,Agricultural—USDA);

association with other product items (e.g., by incoming shipment, byrecipe, by meal kit, others);

packaging status (e.g., none/loose, paper packaging, plastic packaging,vacuum packaging, modified atmosphere packaging);

recommended shelf life.

One or more of the data types described above, may be further correlatedwith customer data that is also stored in the database. Examples of suchcustomer data can include but are not limited to:

dynamic pricing data (e.g., per-quality pricing, coupons, bulkdiscounts);

data of product image offered to customer as part of selection process;

user feedback;

producer preferences;

repeat ordering trends;

temporal ordering trends (e.g., time of day, seasonal, annual, holiday,others);

customer location;

customer diets;

customer menus;

customer health (e.g., allergies);

inventory of items already available in customer's home.

One or more of the data types just described may be further correlatedwith system data that is also stored in the database. Examples of suchsystem data can include but are not limited to:

tray identifier;

row-within-tray identifier;

position-within-row identifier;

carousel identifier;

carousel storage condition(s);

bag identifier;

storage time;

dispense time;

inter-carousel transfer time(s).

Thus according to some embodiments, the database could be arelational-type database having rows corresponding to individualincoming product items that are being handled, and various columnscorresponding to different properties of that item (e.g., source,freshness state, carousel conditions, many others as mentioned herein).Such a database structure allows the data relevant to each handledproduct item to remain associated with that product over the entirelifetime of the handling process (e.g., from intake, through storage,retrieval, and final dispensing to the customer).

As mentioned above, embodiments may employ artificial intelligence (AI)and machine learning techniques—for example to assess product freshnessand predict future freshness based upon inspection and other data types.Such AI approaches involve a specialized procedure executed by aprocessor, which recognizes correlations between a corpus of knownstarting conditions and resulting historical outcomes. The specializedprocedure is trained and refined based upon the historical outcomes, andis then used to predict outcomes for new starting conditions notpreviously encountered (e.g., a new product being received for imaging).

FIG. 32 shows a simplified flow diagram illustrating a machine learningprocess 3200. At 3202, a training data corpus is selected. That trainingdata represents the various historical inputs, and the correspondingknown results.

At 3204, a model for predicting the results from the correspondinginputs is created. That model is mathematical in nature, and can be inthe form of weighted equations, neural nets, and others.

The model is configured to receive inputs in numerical form.Accordingly, at 3206 preprocessing is performed upon the various inputsof the training set, in order to convert them into numbers recognizableby the model. Such preprocessing can involve a variety of techniquessuch as normalization, tokenization, and others.

At 3208, the training data is applied to the model to refine itsstructure. In certain embodiments this training phase may involve theadjustment of weights within the model, to allow it to better reflectthe historical results that are produced from the known inputs. Thesemodel outputs are also in numerical form, and typically representlikelihoods of a plurality of outcomes. In this manner, the model istrained by the training data.

At 3210, a new input is applied to the model that has been trained bythe training data. In response, the trained model outputs a probabilitythat the new input corresponds to a particular outcome. This applicationof new data to the trained model is known as the inference phase.

At 3212, the numerical results output by the model are mapped intouseful information. For example, the mapping to an outcome may correlatea numerical result into to a particular product freshness state.

At 3214, the accuracy of the new predicted output can be evaluated. Thisevaluation of accuracy can then be fed back 3216 into the trainingcorpus to help further refine the accurate performance of the model inpredicting outcomes from new inputs. This new input is applied to themodel as trained by the training data.

FIG. 33 shows a simplified diagram illustrating the implementation ofartificial intelligence principles to product handling according toembodiments. System 3300 comprises imaging camera 3302 that performs anoptical inspection upon incoming perishable product 3304 (here anapple). That optical inspection may be for visible wavelengths, multiplewavelengths (multi-spectral imaging), or may include other than visiblewavelengths (hyperspectral imaging).

The optical data resulting from the optical inspection is stored indatabase 3306. As described previously, other data may also be stored inthe database, for example incoming product source and transportationdata 3308.

Based upon this stored data, an assessment of the physical state 3309 ofthe product may be produced by an inspection artificial intelligencemodel. That state may reveal blemishes or other features characteristicof product freshness (e.g., water content).

The product is then routed by automation control 3310 to a particularcarousel 3312 based upon considerations such as its freshness, expectedshelf life, and popularity. A separate AI degradation model 3314 maypredict the progression of the stored product through its freshnesslifecycle (e.g., as harvested, pre-ripe, ripe, overripe, not usable).This degradation model can ensure that only products of an accuratelypredicted freshness, are ultimately dispensed to the customer accordingto his or her preferences.

In particular, FIG. 33 also shows a customer input 3320 received by afulfillment engine 3322. The engine processes the input, and in responsecauses the carousel to dispense the product at its known state offreshness according to the AI model. As described above, the fulfillmentengine may utilize data including learning feedback 3324 (e.g., from thecustomer) in order to enhance the accuracy of this product dispensingprocess.

As described herein, artificial intelligence and machine learningapproaches may prove useful in predicting a number of different type ofoutcomes in the overall product handling process. One example is usingautomated quality prediction procedures and models in order to separateStock Keeping Units (SKUs) into quality groups that a consumer canselect from between based upon their preferences. Actual images of theindividual items may be offered to the customer during this productselection process.

Artificial intelligence and machine learning approaches may also beutilized in connection with dynamic pricing of specific items. That is,automated quality prediction procedures and models can be used todynamically price each individual item based upon properties such asfreshness, expected expiration, and demand.

Automated quality prediction models may also be referenced in order torank the individual product items belonging to a SKU, by their ripenessand expiration dates.

In summary, the following table lists several AI models, together withtheir expected inputs and outputs.

AI Model Input Output Inspection Optical data Product Blemishes Otherinspection data Product Water Content (e.g., detection of mold,ethylene, etc.) Freshness Optical Product Freshness State LifecycleProduct Blemishes Expiration Date Source/Transport data Other inspectiondata (e.g., detection of mold, ethylene, etc.) Product Freshness StateDynamic Product Price Pricing Customer Data Product Multiple ProductSKUs Multiple Product SKUs Ranking Multiple Product Grouped by FreshnessState Freshness States Product Freshness State Location of Product forStorage Storage Conditions storage in Carousel and Other Storage Factors(e.g., Tray of Product Handling allergies, typical System purchase withother products) Popularity (e.g., possible bottlenecks) Expected DepthLocation in Bag Product Freshness State Bag for Receiving DispensingProduct Ranking Dispensed Product Customer Preferences Expected DepthLocation in Bag

Successful application of various such AI approaches may help to ensurethat a product exhibiting the quality desired by a customer, can berapidly and efficiently delivered to that customer. For example, theconsumer may be offered his or her selection of SKUs in a way thatallows the customer to always consume the item at peak ripeness. Thiscan lead to increased customer satisfaction.

Product handling systems according to embodiments, may provide thecustomer with the ability to purchase a set of individual product itemsbelonging to a particular SKU, with each individual item turning ripe onthe day requested by the customer.

Embodiments may also provide the customer with the ability to purchaseindividual product items belonging to a complementary set of SKUs. Thatis, items belonging to certain SKUs may commonly be purchased together(e.g., cake ingredients), and the system facilitates handling anddelivery of a set of those SKU items together.

Embodiments also allow for dynamic pricing of handled items. Bygenerating and storing in the database an up-to-date and accurateassessment of product quality and expiration date for each item, theitems can be dynamically priced for sale. For example, aging producehaving an earlier expiration date can be dynamically priced at a lowervalue, allowing for more rapid sale and product turnover, and reducingwaste.

Product handling approaches according to embodiments can exhibitincreased efficiency and flexibility in other ways. For example, thelarge number of data types stored in the database and available tovarious AI models, can permit automatic filtering of a consumer'sproduct choices or meal kit choices. This filtering can be based uponstored factors such as consumer diets, consumer allergies, and currentavailability of bulk items in stock at a consumer's home.

As described above, product handling methods and apparatuses accordingto embodiments, may be particularly useful for delivering perishableitems. In some embodiments the shelf life of a product may be measuredin days, for example in the case of fresh produce. Other products mayhave lifespans that are significantly longer than a week, for examplepre-packaged products or aged items such as cheese and wine. Examples ofproduct lifespans may be accommodated according to embodiments are 1day, 2 days, 3 days, 4 days, 5 days, 6 days, 1 week, 2 weeks, 1 month, 6months, 1 year, and 5 years.

While the above description has focused upon the handling and storage ofperishable items, embodiments are not limited to such examples.Alternative embodiments can store non-perishable goods, e.g., thoseexpected to have lifetime of five years or greater.

Clause 1C. A method comprising:

receiving a perishable product item;

sensing an optical image of the perishable product item, the opticalimage comprising a plurality of pixels;

calculating color properties for at least some of the plurality ofpixels;

storing the color properties in a database; and

predicting a freshness state of the perishable product item based uponthe color properties, by referencing a freshness model trained accordingto historical freshness data.

Clause 2C. A method as in Clause 1C further comprising:

identifying a blemish on the perishable product item based upon thecolor properties, by referencing an inspection model trained accordingto historical blemish data; and

storing the blemish in the database.

Clause 3C. A method as in Clause 2C wherein the freshness model furtherconsiders the blemish in predicting the freshness state.

Clause 4C. A method as in Clause 1C further comprising:

loading the perishable product item into a carousel maintained under anenvironmental condition based upon the freshness state; and

storing the environmental condition in the database.

Clause 5C. A method as in Clause 4C further comprising:

controlling the environmental condition based upon the freshness stateto prolong a freshness of the perishable product item.

Clause 6C. A method as in Clause 4C further comprising dispensing theperishable product item from the carousel by,

causing a product conveyor to protrude through an opening in the tray tocontact the perishable product item, and

advancing the conveyor to cause the perishable product item to fall intoa bag having a bottom positioned at a first height proximate to theconveyor.

Clause 7C. A method as in Clause 6C further comprising:

lowering the bottom of the bag away from the conveyor prior to receivinganother product item dispensed on top of the perishable product item.

Clause 8C. A method as in Clause 6C wherein the dispensing is inresponse to a consumer input accepting a price of the perishable productitem.

Clause 9C. A method as in Clause 8C further comprising:

dynamically calculating the price according to a price model based uponthe freshness state.

Clause 10C. A method as in Clause 8C further comprising:

prior to receiving the consumer input, separating the perishable productitem into a freshness grouping based upon the freshness state and aconsumer preference stored in the database.

Clause 11C. A method as in Clause 8C further comprising:

prior to receiving the consumer input, displaying the optical image tothe consumer.

Clause 12C. An apparatus comprising:

a product handling carousel storing a perishable product item under anenvironmental condition;

a database storing a freshness state of the perishable product, acarousel identifier, and the environmental condition;

a fulfillment engine configured to reference the freshness state and thecarousel identifier in response to a customer input, and cause thecarousel to dispense the perishable product item into a bag.

Clause 13C. An apparatus as in Clause 12C further comprising a framehaving forks extendable to retrieve a tray from the product handlingcarousel.

Clause 14C. An apparatus as in Clause 12C further comprising a productconveyor configured to,

protrude through an opening in a tray to contact the perishable productitem, and advance to cause the perishable product item to fall into abag having a bottom positioned at a first height proximate to theconveyor.

Clause 15C. An apparatus as in Clause 14C wherein the frame is moveableto lower the tray over the product conveyor.

Clause 16C. An apparatus as in Clause 14C further comprising a moveableplatform configured to lower a bottom of the bag upon receiving theperishable product item.

Clause 17C. An apparatus as in Clause 12C wherein:

the perishable product item exhibiting an approximately circular crosssection is supported on a tray of a first type; or

the perishable product item exhibiting an approximately rectangularcross section is supported on a tray of a second type.

Clause 18C. An apparatus as in Clause 12C further comprising a cameraconfigured to:

sense an image of the perishable product item, the image comprising aplurality of pixels; calculate color properties for at least some of theplurality of pixels; and store in the database, the color propertiesassociated with the perishable product item.

Clause 19C. An apparatus as in Clause 18C wherein the database isconfigured to store the color properties together with at least one ofsystem data, customer data, and incoming data associated with theperishable product item.

Clause 20C. An apparatus as in Clause 18C wherein the fulfillment engineis configured to:

process the color properties to generate a freshness state for theperishable product item; and

store in the database, the freshness state associated with theperishable product item.

Further details regarding the inspection of products to determinedefects, are now described. Embodiments may use machine learning incombination with inspection to detect defects in produce based on theirarea and defect type.

There are at least two ways to inspect for defects. A first approachinvolves developing a model for each variety of produce. This amounts toaround 700 different models.

An alternative approach develops a universal model for each defect.Examples of such universal models could comprise a bruise model and acut skin model. This defect-based approach results in only about 40models being needed, reducing development time and enhancing robustness.

Specifically, utilizing visual/near-infrared imaging in combination withmachine learning, allows object detection for product item quality.Camera systems are used to take images of produce in a controlledlighting environment.

The images cover wavelengths in the visible RGB range as well as in thenear infrared (NIR) wavelength range. This can be accomplished withmultiple cameras (e.g. one RGB camera and one NIR camera), or a singlemulti-channel camera.

Specific filters can be added to the cameras to focus the inspection ona particular band of wavelengths, especially for the NIR camera. Each ofthe image modalities may reveal different types of defects on produce.For example, FIGS. 34A-B show NIR images, and FIGS. 34C-D show RGBimages.

The acquired images are labeled, and then used to train machine learningmodels to detect defect(s) that may be present. The models can betrained in at least two different ways.

According to one approach, a model is trained to detect all possiblekinds of defects on a single type of item (e.g., apples), by using atraining set comprising known defects on a series of images taken ofthat one item type.

According to another approach, a model is trained to detect a specificdefect (e.g., a bruise) across many types of produce (e.g. as found onapples, pears, tomatoes, peaches, etc.) using a training set comprisingimages including the specific defect on a variety of produce types.

The object-detection type model is one possible example. Here, theRetinaNet deep learning algorithm was adapted for this purpose. The sizeof the resulting bounding box around the detected defect, indicates thesize of the defect.

The resulting information on differentiated defects in the images, andthe size of each defect, is combined with a produce-specific algorithmto determine a quality grading for the item.

Moreover, this information can also be combined with other information(including harvest date, prior storage conditions, future storageconditions, etc.) in a procedure which predicts how the produce willripen or degrade over time.

While the above has described inspection of items utilizing a camera,this is not required. As an alternative or supplement to a camera, aspectrometer may also be used.

Specifically, spectrometer measurements (e.g., reflection, absorbance,transmission, etc.) of an item in the VIS/NIR wavelength range(˜350-2500 nm), permits analysis of the external/internal properties ofthe item, which can include but are not limited to:

color;

external defects;

internal browning;

dry matter content;

sugar content;

firmness;

rot; and

others.

In one implementation, portable spectrometers have been developed foragricultural applications in inspecting produce before picking, or forlater handheld inspection of individual pieces of produce.

An embodiment of a process according to a particular example, may takespectrometer VIS/NIR measurements of incoming produce items. Thatspectrometer data may be correlated with quality, defects, and currentripeness level of produce. FIG. 35 shows a simplified spectrograph.

The spectrometer ripeness prediction is combined with data from othercamera inspection systems and historical information on produce (e.g.,date picked, variety, growing location, previous storage conditions,storage conditions while in grocery store). This data may be used toassess optimal time to deliver produce to customer and predict timelinefor how produce will ripen and/or decay while in the grocery store. Thisapproach reduces waste, improves operations/logistics, and improvescustomer satisfaction.

Products may be inspected prior to entering storage in the carousel(s).Product lifetime may be estimated based upon the inspection results.

In some embodiments, however, products can be automatically inspectedafter storage, either exclusively or in combination with inspectionprior to storage. Such post-storage inspection can employ one or more ofthe following techniques.

Cameras may be used to inspect a full tray or products directly afterbeing pulled from the carousel. Cameras may also inspect individualproducts after they are removed from a tray and before they aredispensed into a bag.

Moreover, sensors other than visual sensors (e.g., cameras) may beemployed to inspect the products. For example, berries degrade quicklyby getting moldy—a negative experience for the customer.

It is possible to sense mold by “sniffing” the gas around the product.In once embodiment, a nozzle could just be positioned to suck air frominside a clamshell enclosing a berry. The sampled air could then beanalyzed. In another approach each case could go through a closed boxwhere the air is analyzed for mold spores.

Details regarding apparatuses that may be utilized for productinspection, are now provided. A variety of methods may flip and/or spinproduct items during camera inspections to allow interrogation frommultiple sides.

Product item shapes may be generally classified into spheres, cylinders,and cuboids. After inspection, the items are sorted into trays based onvarious criteria. A number of approaches may be used to place items ontotrays as part of intake, inspection, and/or sorting of individual uniqueproduct items incoming to the system.

One approach utilizes a pivoting belt conveyor. FIGS. 36A-B showperspective views of a single conveyor having a linkage that can beangled from flat over a range of angles.

Items can translate through the conveyor. FIGS. 37A-F show simplifiedviews of pivoting belt conveyor flows.

Spheres and cylinders will roll when the conveyor is rotated up at anangle. This is shown in FIGS. 37A-C.

Also, the unit can fold over itself in order to flip a box item. This isshown in FIGS. 37D-F.

It is noted that two O-ring belt conveyors can convey almost all items.And, cameras from 6 directions take images of the items. FIGS. 38A-Dshow various views of an embodiment of such a simple convey withmultiple cameras approach. Using six cameras as shown, all sides of theitem can be imaged quickly.

Certain approaches may utilize free fall imaging of items. That is, asitems fall off of a conveyor, they are imaged in the air. FIGS. 39A-Bshow different views of such an approach.

Multiple cameras capture all sides of the items. The item is caught witha cushioned conveyor or other soft landing pad, before moving theproduce further along the conveyor. This method allows for quick imagecapture of all sides of an item, without having to deal with otherhardware being in the camera field of view. This permits a singlemachine learning model to be used with the images taken from allcameras.

Another possible approach utilizes a star wheel conveyor, as shown inFIG. 40. The star wheel conveyor is a multi-arm conveyor that canrotate. FIGS. 40A-C show simplified views illustrating that spheres andcylinders can be rotated as the conveyor belt runs. When in the 12o'clock position, the item can be imaged while rotated.

Cuboid items can be flipped when going through the system. This is shownin FIGS. 40D-F.

Such a star wheel conveyor allows many items to be completely imagedwith a limited number of cameras. The cameras may have the samemechanical components in the FOV, which simplifies the machine learning.

Still another approach may utilize a popup roller conveyor. FIG. 41Ashows a top view, and FIGS. 41B-C show end views of such a mechanism.

According to this approach, two o-ring belts convey items in the store.The rollers pop up next to the conveyor when the item is in place.Cylinders and spheres are rolled along the conveyor, thereby presentingall sides of the item to a few cameras. Different size rollers may beused based on the item size.

This system allows for a majority of items to be fully imaged with alimited number of limited cameras. The cameras may have the samemechanical components in the FOV, which simplifies the machine learningstrategies.

Another mechanism that may be useful for handling items within thesystem, is a roller and spinner conveyor. FIG. 42A shows a top view, andFIGS. 42B-C show perspective end views of an embodiment utilizing thisapproach.

Here, the conveyor comprises trapezoidal rollers, and items are carriedalong by the conveyor. The roller/spinner conveyor stops and the rollersspin, thereby rolling cylinders and spheres. Again, this action allowssides of the item to be presented to only a few cameras. The cameraswill have the same mechanical components in the FOV, which simplifiesthe machine learning.

FIGS. 43A-45C show other possible approaches for handling items forpositioning onto a tray. These methods, described below, can be utilizedalone or in various combinations.

A pop through conveyor offers a possible approach for item handling.

FIGS. 43A-B illustrate top and side views, respectively, of such amechanism. Here, the tray moves left and right while pop throughconveyors advance items along a belt.

Specifically, this concept provides an auto tray loading concept that issimilar to the dispense station. Two belt conveyors distribute the itemsalong the length of each row. The tray moves so the items are located inthe correct row. This approach allows items of different grades to beseparated in their distinct row of the tray, without requiring a buffersystem or unique classification separation system.

Still another item handling approach may utilize an XYZ gantrymechanism. FIGS. 44A and 44B show simplified top and side viewsrespectively, of an embodiment according to this approach. In thesefigures the tray moves left and right and in and out. The item feeddrops item into the tray.

Here, the trays are disposed on a moveable XYZ gantry, and move the trayto a distinct position so the item may be placed in the exact positiondesired. This concept allows items of different grades to be separatedin their distinct row of the tray without needing a buffer system orunique classification separation system.

Still other item handling approaches may utilize a vertical stack buffersystem. FIG. 45A shows a side view, and FIGS. 45B-C show top views,illustrating an embodiment.

Specifically, this approach offers a post-inspection, pre-tray loadinggrade and size filtering system, allowing items of the same grade and/orsize to be are grouped together. This mechanism allows for a simplertray loading mechanism as all items are filtered into the correctcategory, prior to being taken to the tray loader.

In the specific approach illustrated in these figures, there is avertical buffer that builds each row. So, the infeed becomes morecomplex, moving up and down and left to right to feed a fixed trayposition.

As shown in the top view of FIG. 45C, the conveyor branches to allowFirst In, First Out (FIFO) filtering amongst each branch. Each branchcontains the same size and/or grade.

This multiple conveyor setup shown in FIG. 45C, allows for a simplertray stocking mechanism, as all items are filtered into the correctcategory prior to being taken to the tray loader. This embodiment alsooffers relative simplicity from a mechanical perspective.

As has been previously discussed, embodiments may call for items to bestored under controlled conditions, in a carousel structure. Variousaspects of possible designs for the carousel element, are now described.

Initially, it is noted that trays can be shifted from one carousel toanother. This may be done automatically in order to control ripeness.

For example, an avocado should be initially stored in a cold carousel inorder to preserve its life. Then, a few days prior to being dispensed,the avocado should be shifted to a different carousel that is maintainedunder warmer temperature conditions, and paired next to an ethyleneproducing item. This will initiate ripening.

Once the ripening process has commenced, the avocado could be shiftedback to a colder carousel to keep it from over-ripening. The avocado canthen be sold to a consumer at the desired ripeness. Such acarousel-shifting approach allows more products to be stored ininventory, without spoilage and loss of desirable characteristics (e.g.,flavor and nutrients in the case of a produce item).

FIG. 46 shows a simplified perspective view of a front of a carouselaccording to an embodiment. As previously illustrated, items are storedwithin the carousel upon moveable (e.g., vertically moveable) carriers.

FIG. 46A illustrates a perspective view of one possible embodiment of acarrier. This carrier embodiment features lower support beams.

FIG. 46B illustrates a perspective view of an alternative embodiment ofa carrier. Here, the lower supports are replaced with backside supportbeam(s). Such a back-support approach may effectively increase thedensity of items that may be stored within the carousel.

Specifically, by avoiding the space occupied by a bottom support beam, acarrier design can fit 7, 6, 5, or 4 shelves. Carriers of variouscapacities may be available in each carousel, as needed.

FIG. 46C shows a simplified front view of a carrier. Here, two 3 mm backsupport beams take most of the load, and transfer to outer supportwalls. The back beams support vertical load, and resistdeflection/twisting.

Extending out of the page in FIG. 46C, is a single part featuring tabsto hold trays. By eliminating the bottom supports (and increased backsupport) this embodiment affords enhanced vertical tray stackingdensity—according to some embodiments thereby allowing one more fulllayer of trays per carrier. Particular embodiments may offer 10.2% moreproduct storage because of the back (rather than bottom) support design.

Various configurations may further enhance densities of verticallystacked trays within a carousel. For example, while the specificembodiment of FIG. 46B offers support for multi-layers for trays with afixed pitch, that pitch may not be optimal for all items.

Different types of carriers may be present in each carousel. Eachcarrier can be constructed to handle different level counts in order tomaximize density.

For example, FIG. 46D shows carriers having a different number oflevels, and per-level pitches, to allow storage of different items. Asshown in FIG. 46D:

a carrier having 4 levels might offer a greatest stacking density fortrays storing honey dew melons and 2 gallon milk jugs.

a carrier having 5 levels, might offer a greatest stacking density fortrays storing green peppers and mayonnaise jars

a carrier having 6 levels, might offer a greatest stacking density fortrays storing oranges and pasta boxes.

a carrier having 7 levels, might offer a greatest stacking density fortrays storing apples and rice boxes

a carrier having 8 levels, might offer a greatest stacking density fortrays storing limes and salad dressing bottles.

Once items are stored within the carousel upon a tray supported by acarrier, individual items may be dispensed therefrom. Various detailsregarding particular embodiments of dispensing mechanism are nowdescribed.

FIG. 47 shows a perspective view of a front side of a carousel, havingattached thereto a dispensing station according to an embodiment. Theview of FIG. 47 may usefully be contrasted with the previous view ofFIG. 46.

An item dispensing station according to embodiments may feature one ormore elements to enhance performance. For example, it is important toavoid items falling off the dispense station. Accordingly, certainembodiments may feature a dispense conveyor backstop on the pop throughconveyor, in order to keep items from falling backwards.

FIG. 48A shows an enlarged view of a tail on the conveyor, whichprevents items from rolling off the back when the tray is lowered overit and items from a row are dispensed. FIG. 48B shows a simplifiedperspective view of the pop-through conveyor of the dispense station,poised to dispense item(s) from a particular tray row.

Dispense stations according to embodiments may also feature sensors fordetecting dispensed items. FIG. 49A shows a side view of a pop-upconveyor mechanism including an optical sensor for dispensed itemdetection. FIG. 49B shows an enlarged view of the optical sensor in thepop-up conveyor.

This presence sensor detects when an item arrives at the end of theconveyor. The length of the item is already known, so once the item isdetected the conveyor can be driven the known amount to dispense theitem off of the conveyor end. The sensor also detects that the item hasleft the conveyor, thereby allowing reliable confirmation of dispensinga single item.

Weight can also be detected as a sensed quantity during dispensing. FIG.50 shows a perspective view of a dispense station configured with loadcells for this purpose.

Load cells may permit weight measurement to detect items falling off thedispense station (e.g., off of the back). Load cells can also be used todetect the change in weight as an item leaves the conveyor.

Load cells may be positioned in a number of different possiblelocations, including but not limited to:

under the mounting rails for the mobile platform;

on the extending conveyor rail; and/or

at a strain gauge between the end rollers.

As an item is dispensed off the end of the conveyor, the load cellreading marks a step change, indicating the reduction in mass on theconveyor. This again permits reliable dispensing of just one item at atime from the tray.

In order to accurately accomplish item dispensing in a reliable manner,it may be useful to precisely control tray position. Maintainingpositional control of the tray within a tolerance, allows the dispenseconveyors to repeatedly fit through the slots in the tray, whilereducing the layers of closed loop motion control.

The exercise of positional control over the tray may commence when thetray arrives at the carousel (FIG. 51A), and is then loaded into thecarousel (FIG. 51B). From the time the tray is initially physicallycontacted, it is passed from pinned location to pinned location.

FIG. 51C shows a simplified view of the use of indexing pins.Specifically, the tray lifter aligns with the tray and lifts it up withthe end effector. These end effectors on the tray lifters inserting andremoving the tray from the carousel, have pins that index with the tray.

The carrier has pins that index with the tray. The tray arrives at theupper tray lifter and is lifted by the end effector. The end effectorpins index with the tray, locating it precisely. The upper tray lifterinserts the tray into the carrier, where it also indexes on pins. Usingsuch passive (e.g., pin) features to control position of the tray duringstorage, allows for closer tolerances on the dispensing end effectorwhile still allowing retrieval without closing the loop between the endeffector position and the tray position.

Exercising positional control of the tray on the end effector, alsopermits precise disposition of the dispensing conveyors so they fitthrough the slots in the tray without closing the loop between trayposition and dispensing conveyor position.

Details regarding designs for the end effector are now described. It wasdiscovered that reaching relatively deep into a narrow space called forbetter control over the angle of the end effector, than:

reaching into a short space, or

not reaching in at all, and just hooking the front end of the tray.

Accordingly, FIG. 52A illustrates a simplified side view of an endeffector design according to one embodiment. Here, using injectionmolded parts for the trays allowed for a simplification in the carrier.The shelf upon which the tray sits, was made flat rather than a U-shape,in order to dodge the tray lifter linear slides.

FIG. 52B shows an alternative embodiment wherein the end effector on thetray lifter, is a hook that engages with the front edge of the tray. Thetray is pushed into or pulled out of the carrier, by the hook endeffector.

The tray slides on the carrier shelf and also slides on a shelf on thetray lifter. This eliminates the issue of leveling the carriers to thetray lifter and maintaining level tolerance for every carrier in everyloaded state.

FIG. 52C shows another alternative embodiment. Here, the end effector onthe tray lifter may have a pinion gear that engages with a rack (whichmay be molded into the tray). Driving the pinion gear pulls the tray outof the carrier and onto the tray lifter. This eliminates the problem ofleveling the carriers to the tray lifter, and maintaining a tight leveltolerance for every carrier in every loaded state.

FIGS. 53A-B show different views of carrier design embodimentsconfigured to interact with the end effectors of FIGS. 52B-C. Here, withthe new hook design or gear rack design, the slides do not have toinsert into the carrier.

This allows for increasing the pitch of the carrier shelves receivingthe tray. Such increased shelf pitch promotes an increase in traypacking density, by allowing choice of the best shelf to put the tray onto allow it to be closest to the next tray above or below.

It is noted that the slides holding the end effector need to fit betweenthe shelves. Not needing to preserve open space for slides, allows theshelf pitch to be tighter. The tray can be positioned closer to the oneabove or below it depending on the height of items in the tray.

Details regarding various designs for the customer bag and thetransporting same for shipping, are now discussed.

A traveler containing a bag to receive dispensed items, may feature aplatform that can be raised and lowered. There are several possible waysto raise and lower the platform.

FIG. 54A shows a perspective view of one embodiment of a traveler. Inthis embodiment, the traveler platform may be moved up and down throughthe operation of lead screws that are driven by a chain or belt.

FIG. 54B shows an alternative embodiment of a traveler. Here, theplatform may be raised or lowered utilizing linear actuator(s), andguide rail(s) located in opposite corners.

Alternative designs for travelers are of course possible. Certainembodiments could feature the platform attached to a scissor lift. Someembodiments could feature the platform lifted in corners by belts orpulleys. Also, a traveler design could comprise an assembly of springsthat compress when the bag is full, and extend when the bag is empty.

Some embodiments may add load cell(s) to the platform, allowingmeasurement of changes in weight at the traveler. FIGS. 55A-B show viewsof an embodiment of a traveler incorporating load cells.

In this embodiment, the load platform comprises two plates. The bottomplate is anchored to lead nuts that travel up and down the screws.

Four cantilever beam load cells support the top plate. These load cellsmeasure the weight of items in the bag, helping to determine if itemswere successfully dispensed.

While the specific configuration of load cell may vary, the basicconcept features the measurement of weight changes to evaluatedispensing. Given known item weights, with proper calibration thedifference between weight changes (e.g., 1 apple vs. 2 apples) canreveal when possible errors occur.

Bag(s) for housing items dispensed from the carousel, are supported bythe traveler. One or more methods may be utilized to hold the bag inplace at the top of the frame.

FIGS. 56A-B show top perspective, and enlarged views respectively, of atraveler including a frame. Bags are first bunched or accordioned,around a rectangular box for quick loading into the traveler. Thisrestrains fabric to the perimeter of the bag, thereby leaving the bagbottom exposed to receive the dispensed item, without interference frombunched fabric.

According to particular embodiments, the bags may have fabric grommetsinstalled in their upper corners. Bags are loaded into the traveler byhooking the grommets over bent pegs located in the upper corners of theframe.

A hinged lid (with an integrated latch) opens and closes to lock the bagin place. The inner rim of the lid may be lined with bulb edge guards toprotect dispensed items from being damaged.

However, alternative designs for the traveler are possible, with the bagbeing fixed according to other approaches. For example, FIG. 57A shows atraveler embodiment featuring straight vertical grommet pegs, stickingup. An (optional) removable rubber cap may be tethered to the bottom ofthe peg, for a quick locking feature with a part that doesn't get lost.

FIG. 57B shows an alternative traveler embodiment featuring straightvertical grommet pegs, sticking down, embedded in the lid. When closed,the pegs insert into holes, locking the bag in place.

Still further alternative traveler designs are possible. Someembodiments may feature a quick turn arm and locking post. Otherembodiments may include velcro tabs sewn on to the upper bag corners.Magnets could also be used as clamps.

FIGS. 58A-B show simplified views of a plunger and grommet, and travelerembodiment incorporating same. Here, pop-out panel fastening plungersare installed in bag rim to lock into place. A loop of sheet metal wrapsaround grommet and bag fabric, clamping the bag in place.

A traveler according to an embodiment may feature a frame topped with ahinged lid. FIGS. 59A-C show simplified views of a hinged lid. Theenlarged view of FIG. 59C shows the T-profile hinge integrated intosheet metal, locked in place after initial assembly by installing ablock covering the wide slot. FIGS. 60A-B are perspective views of atraveler showing the hinge and spring clasp, with the lid open andclosed, respectively.

According to certain embodiments, the traveler may be powered by aquick-swappable battery pack design that can be easily replaced forcharging. The battery pack may comprise a housing and two sealed leadacid batteries wired in series inside the housing.

FIG. 61A shows a battery pack embodiment with the power plug located inthe back of the housing relative to the front flange. FIG. 61B shows abattery pack embodiment with the power plug located on the side of thehousing relative to the front flange.

A processor can measure the battery's state of charge by reading itsvoltage. When the battery pack is low on charge, it can be removed fromthe traveler base without tools. It may be unplugged without any loosewires; the power plug and jack are embedded in the traveler base andbattery pack housing, respectively.

The battery pack may be charged in a parallel rack system located closeto the bag unloading station. A fully charged battery can be loaded intothe traveler so it can continue its work with only a short interruption.

As noted previously, particular embodiments may afford movement of thetraveler based on conveyor action. FIG. 62 is a simplified view showingconveyors 6200 can stop tray 6201 in front of an individual stagingposition 6202, to receive item(s) 6204 dispensed from a given carousel6206.

In particular embodiments, the conveyor system for the traveler maycomprise a collection of modular conveyor sections and 90° transferstations. This conveyor system moves the travelers around the fulfilmentcenter to accomplish their tasks (e.g., receiving dispensed items,possibly loading trays into carousels).

The conveyors may be arranged in grids and loops. Each section iscapable of running in both forward and reverse directions.

According to embodiments, each conveyor section is an independentlycontrolled set of rollers linked together with drive O-rings. One rollerpin in each conveyor section has an integrated motor with servo controlthat allows precise positioning of a traveler on that section, in thedimension of conveyance (e.g., across the dispensing station located atthe front of the carousel).

Position control along this dimension, combined with position control ofthe dispense arms of the tray lifter in a perpendicular dimension,allows items to be dispensed to precise locations in the bag.

When a traveler is supported by rollers of adjacent conveyor sections,control software may instruct coordinated movement of both conveyorsuntil the traveler is fully in control of the downstream section. Atthat point, the section can move the traveler around without affectingthe positions of other travelers.

This independent control can facilitate precise item dispensing at everydispense station. A photoelectric sensor may be integrated in eachconveyor section to trigger a reference position of the traveler as itmoves onto the section.

Different variations on the design and operation of the travelercomponent, could serve to enhance system operation. For example, incertain embodiments the traveler may include its own drive system (e.g.,wheels, tracks, rails, others) and/or sensors (e.g., cameras,line-following optics) that permit autonomous or semi-autonomousmovement throughout the fulfilment center.

Certain embodiments may feature the addition of duplicate bag lockingmechanisms. These could allow each traveler to carry multiple bags atonce.

It is noted that a single traveler could fulfil a single order, multipleorders, or parts of separate orders, during various trips around thefulfilment center. This can lead to enhanced efficiency.

Certain embodiments may feature the addition of a tray-carryingattachment to the traveler. This would allow the traveler to bring traysto each carousel for loading. Such integration of tray loading and itemdispensing with a common moveable structure (the traveler), would leadto substantially simplified operation.

Some embodiments could feature the addition of a high speed actuatorcapable of moving the load platform downward at high speed. Suchnear-matching the downward projectile velocity of a dropped item coulddesirably reduce impact forces on the item and upon the load platform.

Some specific details regarding design and operation of trays accordingto particular embodiments, are now provided. Trays may be fabricatedutilizing an injection molding approach that ensures high strength.

According to certain embodiments, the entire tray may be created as asingle, unitary injection-molded piece. FIG. 63A shows an end view of atray that is configured to hold circular items.

Here, the largest outer diameter (OD) of a stored item, matches pitch.The smallest OD is determined by a center of a round cross-section nottipping past hump on the center beam of the tray.

The strength of the center beam is dependent upon height. This reducespacking density in the carousel. The base may be widened with ribs inorder to resist side loads

FIG. 63B shows an end view of a tray configured to hold cuboid items.Boxes may also fix in the same rows and rounds. As width of stored itemsincreases, items start to rest upon the lips of side walls. This servesto spread the load while keeping smaller boxes under belts. Conveyorbelts of the dispensing station go up through same position, dependingon tray type and item type (e.g., round vs cuboid). Belts are designedto ensure contact with the smallest allowable box item of a given traytype.

FIG. 64 shows a top view of a tray according to an embodiment. Bumpsstop items from traveling past the curve of the belt. Holes molded infor indexing to the carousel and to the dispense station. Crossing websto add additional support and torsional resistance Webs in outer beam,align with inner beam walls in order to reduce wall deformation.

Reliance upon a single, unitary tray design, can increase manufacturingcost. Moreover, the resulting fixed size of tray features (e.g., numberof rows, width of rows, number and shape of separators) can limitflexibility and degrade efficiency.

Accordingly, alternative embodiments may feature a design that allowsdifferent tray types to be assembled from a plurality of injectionmolded parts. FIG. 65A shows a perspective view of one tray type, andFIG. 65B shows a perspective view of another tray type, that may beassembled from a plurality of parts.

The enlarged tray views of FIGS. 66A-B show 2 types of support beams.One type has mates on both sides. Center beams clip into the supportbeams. There are 4 types of center beams, with a center separation beambeing of one type.

FIGS. 67A-B are enlarged views showing tray features formed frommultiple molds. In particular, the view of FIG. 67A shows the supportbeam fitting over the side beam. Key features keep parts from turning atthe bottom. The view of FIG. 67B shows a pin poking through 4 tabs,bending them back, and locking the pin vertically.

In some embodiments, trays having various different features may beassembled on demand. Equipment may be designed to accomplish this trayassembly process.

Specifically, item handling systems according to embodiments may utilizea fixed number (e.g. 10) of tray types. Each tray type may hold acertain size of item.

Individual trays may comprise a plurality of parts in the form of long,narrow, injection molded pieces that clip together to form a tray.

Following item inspection, the tray loading station needs to receive anappropriate tray type to load the inspected item into.

Rather than store stacks of trays of each type, a machine may access theindividual pieces stored sorted according to type. Then, on demand, themachine may then assemble a tray of a certain size.

It is noted that the unassembled pieces have a higher storage densitythan the assembled tray. Thus, on-demand tray assembly can save valuablespace within the fulfillment facility.

According to embodiments, the machine could also assemble on-the-fly,trays in configurations other than the limited number of standard types.This could be done by building rows with different spacings, as calledfor by the tray loading station.

Empty trays removed from the carousel could be returned to the machine.There, the empty trays are automatically broken back down intoindividual pieces for efficient storage.

Such approaches featuring on-demand tray assembly, may offer one or morebenefits. As mentioned above, these approaches may save space on theline.

Also, on-demand tray assembly may allow for a larger variety of traystypes, and may decrease the distance a tray has to travel to get to thetray loading station if the machine can be located closer (potentialtime reduction).

On-demand tray assembly may reduce the total number of tray piece partsrequired. This is because trays can be reconfigured as necessary, ratherthan wasting tray inventory space on tray types that aren't beingutilized.

Returning now to discussion of general system operation, under somecircumstances it may be desired to dispense an item that is not locatedat the end of a tray row (e.g., is stored in the middle of the tray).Accordingly, certain embodiments may utilize a gantry robot fordispensing purposes, as is shown in the simplified view of FIG. 68.

For example, when each item in a SKU has a different value, it may bedesired to dispense from the center of a tray. Thus where the product isa steak, there may be 40 images of different cuts of steak that areavailable to show the customer for selection. By allowing a gantry toaccess center portions of the tray, all of the steak cuts can beavailable for dispensing.

Another possible circumstance that may be favorable to the use of agantry robot, is where the items are not amenable to conveyordispensing. This could be due, for example, to the items' size, shape,and weight. In such cases, a gantry robot may be a viable alternative.

Also, when items are small and thin (e.g., a gravy packet or beef jerkypouch), it would be advantageous to stack them vertically in one trayand then pick the top item from each tray. This increases density.

Finally, dispensing with a gantry robot could be useful when an item israrely purchased. Under such circumstances, it is inefficient to carryan entire row of that SKU. The use of a gantry robot to pick from thecenter of a tray, allows placement of multiple SKUs in one row, andpicking them out as needed.

It is further noted that a gantry robot could be employed in combinationwith a conveyor. Specifically, a gantry fixed at the end of conveyorbelt(s) could catch the item and lower it into the bag. This couldreduce damage to items.

Item handling systems according to embodiments, may not be limited to asingle physical location. Items may be dispensed and delivered frommultiple smaller sites that are located relatively close to one another(e.g., in different neighborhoods within a single large metropolitanarea.

Under such conditions, an inventory management solution may feature onecentral store (the hub) and many satellite stores (the spokes). Thisapproach could be implemented with tray packaging performed at the hub,and trays shipped to each spoke. Carrying racks could be designed toallow distribution vehicles to rapidly and easily perform loading,transport, and unloading.

In summary, FIG. 69 offers a simplified flow diagram of a processcontrol according to an embodiment. This flow diagram illustrates theinteraction between a central process and control system with variousphases of order fulfillment on the front end, and inventory managementon the back end.

A system according to various embodiments may include one or morefeatures to enhance automated dispensing and delivery of handled items.For example, in order to replace pallet jacks, robotic pallet jacks canbe utilized remove items from trucks that are delivering bulk goods forintake, inspection, sorting, and loading into carousels. Also, roboticssystems that can raise and lower pallets and insert into shelving, maybe relied upon to lift pallets onto shelves. Further, once a pallet isready for inspection, automation equipment may move boxes from pallets,and move the box contents onto a conveyor.

Moreover, particular embodiments may automate order packing as follows.When a full traveler arrives, equipment would lift a bag out of thetraveler and move it to a shipping area. Another piece of equipmentcould place a new bag into the traveler.

A robotic cart would be deployed to transport bags from the factory tothe parking lot. The robotic cart would place bags into the mode oftransportation for delivery.

A variety of delivery options are of course possible. Vehicles could bemanually driven by humans, or self-driven. Alternatives can includedelivery by bikes, carts, drones, or trucks plus drones.

It is noted that various embodiments may enhance a customer's experienceby allowing flexibility and choice in selecting items. A customer maysearch for items on a website utilizing a search bar.

Under this approach, the user enters a product name, and in response ispresented with a list of options. The customer selects an option, whichis added to the customer's cart. While the search bar approach is fastfor individual items, it can be laborious for a cart comprising multipleitems. Also, the search bar approach requires a customer to know desireditems in advance.

In contrast, with the search bar approach, an aisle-based approachinvolves a customer seeking a particular product item, to traverse acomplex hierarchy. Thus, a customer seeking to purchase a red onion, mayneed to:

click fruits and vegetables

click vegetables, then

scroll to find red onions.

Such an aisle-based approach takes longer than searching if a customerknows ahead of time what is desired. But, the aisle approach affords thecustomer the ability to browse different options, and get inspirationfor other products. However, aisle-based selection can be tedious due tothe many possible available options to scroll through in each aisle.

Accordingly, particular embodiments may permit customer searching foritems based upon categories. This approach can specifically leadcustomers to the particular items that they want to buy.

Under the category-based approach, a customer browses item categories,permitting visualization in a more useful manner. The categories wouldbegin as general, and then narrow to become more specific.

For example, FIG. 70 shows a simplified view of categories for fooditems that are organized into a tree hierarchy. Once the category“Pizza” is selected by a customer, the website offers a variety of itemsfor selection.

One possibility is to display for customer selection, standard pizzaoptions (e.g., pizza crust, sauce, cheese). Meat Lovers will add sausageand salami (for example).

Such category-based searching can depict ingredients going into making apizza. So, this approach shows different types of pizza dough (frozen,refrigerated, different brands), the sauces (bottled, cans, brands),different vegetables, etc.

Then, the user could (de)select one or more suggested items, removing orchanging the brand. Finally, the user could add to the cart all of theitems selected by category, and move on to a different category.

Over time, an engine of the system could learn preferences of specificusers/user groups, and suggest appropriate option the next time thepizza category is selected.

Embodiments may suggest ideas to a customer based upon machine learning.Embodiments could suggest specific items for purchase available in stockin large volumes.

Through the user of category-based searching, the consumer's experienceis more rapid and intuitive. For example, in shopping for grocery items,customers traditionally think in terms of item groups of meals andcategories, e.g.:

“We need three dinners tonight, I need breakfast and lunch food for theweek and I wanted to make that chocolate cake on Wednesday for Sue totake to school”.

Here, the user can go to each category and select out their dinners in afew action steps (e.g., clicks). Similarly, a customer can select arecipe for the cake and obtain all the ingredients at once, or they canclick each ingredient from one basic list if they have a recipe.

In this manner, customers are free to select items in a more intuitivemanner, mirroring how they think and in terms of groups of items. Thusinstead of having to choose and select 30 separate items, they areselecting instead only 7:

3 dinners

2 lunch options

1 breakfast option

1 cake option.

Category-based selection can enhance the speed of purchase, movingcustomers through the site faster. Currently it can take around 60 minto fill a virtual shopping cart, a time comparable to (or even longerthan) physically going to a grocery store. Improving the check out timeis can aid in securing a steady on-line customer base.

Category-based item shopping can also enhance the flexibility of theseller. Grocery stores may seek to leverage relations between items byplacing them in physical proximity (e.g., salsa next to chips, marinadesby the meat counter). However, this is done in a crude, opportunisticmanner, constrained by available physical space (it is unfeasible toposition pizza sauce, veggies, pepperoni, and pizza dough, all in a sameaisle).

By contrast, a website offers a virtually unlimited variety of ways togroup related items to facilitate common purchase. Category-basedselection leverages this inherent advantage of web-based shopping, toenhance the user's experience.

As described in detail above, embodiments of systems and methods forhandling items may be particularly suited for implementation inconjunction with a host computer including a processor and acomputer-readable storage medium. Such a processor and computer-readablestorage medium may be embedded in the apparatus, and/or may becontrolled or monitored through external input/output devices.

FIG. 71 is a simplified diagram of a computing device for processinginformation according to an embodiment. This diagram is merely anexample, which should not limit the scope of the claims herein. One ofordinary skill in the art would recognize many other variations,modifications, and alternatives.

Embodiments according to the present invention can be implemented in asingle application program such as a browser, or can be implemented asmultiple programs in a distributed computing environment, such as aworkstation, personal computer, mobile device or a remote terminal in aclient server relationship.

FIG. 71 shows computer system 7110 including display device 7120,display screen 7130, cabinet 7140, keyboard 7150, and mouse 7170. Mouse7170 and keyboard 7150 are representative “user input devices.” Mouse7170 includes buttons 7180 for selection of buttons on a graphical userinterface device. Other examples of user input devices are a touchscreen, light pen, track ball, data glove, microphone, and so forth.

FIG. 71 is representative of but one type of system for implementingvarious embodiments. It will be apparent to one of ordinary skill in theart that many system types and configurations are suitable for use inconjunction with item handling.

According to one example, computer system 7110 includes a Pentium™ classbased computer, running Windows™ XP™ or Windows 7™ operating system byMicrosoft Corporation. However, the apparatus is easily adapted to otheroperating systems and architectures by those of ordinary skill in theart without departing from the scope of the present invention.

As noted, mouse 7170 can have one or more buttons such as buttons 7180.Cabinet 7140 houses familiar computer components such as disk drives, aprocessor, storage device, etc. Storage devices include, but are notlimited to, disk drives, magnetic tape, solid-state memory, bubblememory, etc. Cabinet 7140 can include additional hardware such asinput/output (I/O) interface cards for connecting computer system 7110to external devices external storage, other computers or additionalperipherals, further described below.

FIG. 71A is an illustration of basic subsystems in computer system 7110of FIG. 71. This diagram is merely an illustration and should not limitthe scope of the claims herein. One of ordinary skill in the art willrecognize other variations, modifications, and alternatives.

In certain embodiments, the subsystems are interconnected via a systembus 7175. Additional subsystems such as a printer 7174, keyboard 7178,fixed disk 7179, monitor 7176, which is coupled to display adapter 7182,and others are shown. Peripherals and input/output (I/O) devices, whichcouple to I/O controller 7171, can be connected to the computer systemby any number of approaches known in the art, such as serial port 7177.For example, serial port 7177 can be used to connect the computer systemto a modem 7181, which in turn connects to a wide area network such asthe Internet, a mouse input device, or a scanner. The interconnectionvia system bus allows central processor 7173 to communicate with eachsubsystem and to control the execution of instructions from systemmemory 7172 or the fixed disk 7179, as well as the exchange ofinformation between subsystems. Other arrangements of subsystems andinterconnections are readily achievable by those of ordinary skill inthe art. System memory, and the fixed disk are examples of tangiblemedia for storage of computer programs, other types of tangible mediainclude floppy disks, removable hard disks, optical storage media suchas CD-ROMS and bar codes, and semiconductor memories such as flashmemory, read-only-memories (ROM), and battery backed memory.

As previously mentioned, item handling systems according to embodimentsare not limited to conveyor-type systems. Alternative embodiments couldinclude a drive system for a traveler that is responsible for receivingitems dispensed from a carousel, and even also responsible for movingtrays having items disposed thereon, for loading into a carousel.

FIGS. 72A-B show views of different traveler embodiments featuring adrive system. Such a traveler may feature integration of a drive system,e.g., tracks (FIG. 72A), wheels (FIG. 72B), others, and sensors (e.g.,cameras, line-following optics) in order to allow the traveler to moveitself around the fulfilment center.

Moreover, certain embodiments of travelers may feature multiple bagsadded to a single drive unit, e.g., track (FIG. 73A), wheels (FIG. 73B).Such embodiments may feature duplicate bag locking mechanisms so thateach traveler can carry multiple bags at once. In this manner, a singletraveler can fulfil more of a single order, or parts of separate orders,during trips around the fulfilment center for improved efficiency.

It is further noted that a traveler may be equipped to include a traycarrying capability. The addition of such a tray-carrying attachment mayallow the traveler to bring trays to each carousel. FIGS. 74A-B showembodiments of tracked and wheeled travelers, respectively, carrying atray.

It is noted that a traveler need not be equipped to carry a bag forreceiving dispensed items. According to some embodiments, a travelercould be designed to carry tray(s) only. FIGS. 75A-C show front, frontperspective, and side perspective views respectively, of such a(wheeled) tray traveler robot.

Under certain implementations, there could be two robot types on thefloor of the fulfillment center. One robot type carries customer bagsbetween carousels, filling them with dispensed items as they go. Theother robot type carries multiple (e.g., ˜5-15 full trays), loading theminto carousels as they go. The robots could be environmentallycontrolled to ensure the condition chain is maintained.

Clause 1D. An apparatus comprising:

a first carousel storing a first product under a first set ofenvironmental conditions, the first product corresponding to a firstStock Keeping Unit (SKU);

a second carousel storing a second product under a second set ofenvironmental conditions different from the first set of environmentalconditions, the second product corresponding to a second SKU differentfrom the first SKU; and

packaging disposed on a traveler that is moveable to,

a first station proximate to the first carousel to receive the firstproduct dispensed in a first direction into the packaging, and then

a second station proximate to the second carousel to receive the secondproduct dispensed in the first direction into the packaging, and then

a shipping station proximate to a vehicle for receiving the packaging.

Clause 2D. An apparatus as in Clause 1D further comprising a first trayconfigured to support the first product and a third product in the firstcarousel, the third product corresponding to a third SKU different fromthe first SKU and the second SKU.

Clause 3D. An apparatus as in Clause 2D wherein:

the first tray is configured to support the first product in a firstrow; and

the apparatus further comprises a second tray moveable within the firstcarousel in a second direction proximate to the first dispensingstation.

Clause 4D. An apparatus as in Clause 3D wherein the first direction issubstantially orthogonal to the second direction.

Clause 5D. An apparatus as in Clause 4D wherein the first direction ishorizontal.

Clause 6D. An apparatus as in Clause 5D wherein the second direction isvertical.

Clause 7D. An apparatus as in Clause 2D wherein the first item isdispensed to the packaging from the first tray located within the firstcarousel.

Clause 8D. An apparatus as in Clause 2D wherein the first item isdispensed to the packaging from the first tray moved outside the firstcarousel.

Clause 9D. An apparatus as in Clause 2D wherein the first tray definesan opening receiving a moveable member to contact the first product.

Clause 10D. An apparatus as in Clause 9D wherein the moveable membercomprises a belt configured to move in the first direction.

Clause 11D. An apparatus as in Clause 10D wherein:

the first direction is horizontal; and

the first tray is configured to move vertically for the belt to projectwithin the opening and contact the first product.

Clause 12D. An apparatus as in Clause 10D wherein:

the first direction is horizontal; and

the belt is configured to move vertically to project within the openingand contact the first product.

Clause 13D. An apparatus as in Clause 2D wherein:

the first direction is horizontal; and

the first tray is configured to drop the first product in a verticaldirection off a tray end into the packaging.

Clause 14D. An apparatus as in Clause 13D further comprising a flexiblemember configured to cushion the first product falling into thepackaging.

Clause 15D. An apparatus as in Clause 14D wherein the cushion isdisposed within the packaging.

Clause 16D. An apparatus as in Clause 2D wherein the tray comprises aplurality of injection molded parts assembled based upon dimensions ofthe first product.

Clause 17D. An apparatus as in Clause 1D wherein the traveler includes aweight sensor.

Clause 18D. An apparatus as in Clause 1D wherein:

the first set of environmental conditions comprises a first conditionselected from,

a first temperature,

a first humidity,

a first light level,

a first gas ambient,

a first mold spore level, or

a first ethylene level; and

the second set of environmental conditions comprises a second conditiondifferent from the first condition and selected from,

a second temperature,

a second humidity,

a second light level,

a second gas ambient,

a second mold spore level, or

a second ethylene level.

Clause 19D. An apparatus as in Clause 18D wherein the second carousel isconfigured to receive from the carousel, the first tray supporting thefirst product.

Clause 20D. An apparatus as in Clause 18D wherein:

the first set of environmental conditions is configured to prolong ashelf life of the first product; and

the second set of environmental conditions is configured to accelerate aripening of the first product.

Clause 1E. An apparatus comprising:

a first carousel storing under a first set of environmental conditions,a first product supported on a first tray;

a second carousel storing under a second set of environmental conditionsdifferent from the first set of environmental conditions, a secondproduct supported on a second tray; and

a non-transitory computer-readable storage medium in communication withthe first carousel and with the second carousel to record datacomprising,

a first identifier of the first product,

a first location of the first product,

a second identifier of the second product, and

a second location of the second product.

Clause 2E. An apparatus as in Clause 1E wherein the first locationcomprises:

a third identifier of the first carousel;

a fourth identifier of the first tray; and

a fifth identifier of a position of the first product within the firsttray.

Clause 3E. An apparatus as in Clause 1E wherein the data furthercomprises status information of the first product.

Clause 4E. An apparatus as in Clause 3E wherein the status informationcomprises:

a weight of the first product;

a visual inspection result of the first product;

a non-visual inspection result of the first product;

an image of the first product;

a harvest date of the first product;

a shipping date of the first product;

a receipt date of the first product;

an expiration date of the first product;

a quality of the first product;

a name of the first product;

a brand of the first product;

a supplier of the first product;

a history of environmental storage conditions of the first product;

a moisture content of the first product;

a variety of the first product;

a Stock Keeping Unit (SKU) of the first product;

a regulatory classification of the first product;

a recipe including the first product;

dynamic pricing data; or feedback data.

Clause 5E. An apparatus as in Clause 3E further comprising an engineconfigured to:

receive a first input specifying a particular SKU;

match the particular SKU with the first product; and

provide at least a portion of the status information regarding the firstproduct to a customer.

Clause 6E. An apparatus as in Clause 5E wherein the portion of thestatus information comprises an image of the first product.

Clause 7E. An apparatus as in Clause 5E wherein the portion of thestatus information comprises a quality of the first product.

Clause 8E. An apparatus as in Clause 5E wherein the engine is furtherconfigured to:

receive a second input selecting the first product; and

instruct the first carousel to dispense the first product intopackaging.

Clause 9E. An apparatus as in Clause 8E wherein the engine is furtherconfigured to:

receive a third input selecting the second product; and

instruct the second carousel to dispense the second product into thepackaging.

Clause 10E. An apparatus as in Clause 9E wherein the engine isconfigured to instruct the second carousel to dispense the secondproduct into the packaging before the first product, based upon a pathbetween the first carousel, the second carousel, and a shipping station.

Clause 11E. An apparatus as in Clause 9E wherein the engine isconfigured to instruct the second carousel to dispense the secondproduct into the packaging before the first product, based upon aconsideration comprising a weight, a size, or a fragility of the firstproduct relative to the second product.

Clause 12E. An apparatus as in Clause 8E wherein the packaging comprisesa bag.

Clause 13E. An apparatus as in Clause 8E wherein the packaging isdisposed on a traveler moveable between the first carousel and thesecond carousel.

Clause 14E. An apparatus as in Clause 13E wherein the traveler isconfigured to move on a conveyor.

Clause 15E. An apparatus as in Clause 13E wherein the traveler isconfigured to move on a rail.

Clause 16E. An apparatus as in Clause 13E wherein the traveler isconfigured to carry the first tray.

Clause 17E. An apparatus as in Clause 13E wherein the traveler compriseswheels.

Clause 18E. An apparatus as in Clause 17E wherein the traveler iscapable of semi-autonomous movement.

Clause 19E. An apparatus as in Clause 8E wherein the engine is furtherconfigured to instruct inspection of the first product prior todispensing into the packaging.

Clause 20E. An apparatus as in Clause 1E wherein the first traycomprises a plurality of injection molded parts assembled based upondimensions of the first product.

Clause 1F. A method comprising:

receiving a first product;

performing a first inspection of the first product;

an engine determining from the first inspection, a quality of the firstproduct;

the engine recording in a non-transitory computer readable storagemedium, the quality related to an identifier of the first product;

based upon the quality, sorting the first product into a first traystored in a first carousel under a first set of environmentalconditions;

the engine recording in the non-transitory computer-readable storagemedium, a location of the first item related to the identifier and thequality; and

in response to a first input, the engine offering the first item for acustomer selection including display of the quality.

Clause 2F. A method as in Clause 1F wherein the quality comprises animage of the first product.

Clause 3F. A method as in Clause 1F wherein the quality is determinedfrom a physical appearance of the first product.

Clause 4F. A method as in Clause 1F wherein the quality is determinedfrom a taste of the first product.

Clause 5F. A method as in Clause 1F wherein the quality is determinedfrom a ripeness of the first product.

Clause 6F. A method as in Clause 1F wherein the quality is determinedfrom a nutritional value of the first product.

Clause 7F. A method as in Clause 1F wherein the quality is determinedfrom a physical makeup of the first product.

Clause 8F. A method as in Clause 1F wherein the quality is determinedfrom a lifetime of the first product.

Clause 9F. A method as in Clause 8F wherein the engine is furtherconfigured to predict the lifetime utilizing a model.

Clause 10F. A method as in Clause 9F wherein the model considers one ofmore conditions of the first set of environmental conditions.

Clause 11F. A method as in Clause 1F wherein the first inspectioncomprises an optical inspection.

Clause 12F. A method as in Clause 1F wherein the first inspectioncomprises spectroscopy.

Clause 13F. A method as in Clause 1F further comprising:

the engine determining a price for the first product based upon thequality; and

the engine displaying the price with the quality.

Clause 14F. A method as in Clause 1F wherein the first input comprises aStock Keeping Unit (SKU) of the first product.

Clause 15F. A method as in Clause 1F further comprising:

based on the quality, the engine instructing transfer of the firstproduct to a second carousel under a second set of environmentalconditions different from the first set of environmental conditions.

Clause 16F. A method as in Clause 15F wherein:

the first set of environmental conditions is configured to prolong ashelf life of the first product; and

the second set of environmental conditions is configured to accelerate aripening of the first product.

Clause 17F. A method as in Clause 1F further comprising:

in response to a second input comprising the customer selection, theengine instructing dispensing of the first product into packaging.

Clause 18F. A method as in Clause 17F further comprising:

the engine instructing movement of the traveler and the packaging to ashipping station.

Clause 19F. A method as in Clause 17F further comprising:

the engine instructing movement of the traveler to receive a secondproduct dispensed from a second carousel under second environmentalconditions different from the first environmental conditions.

Clause 20F. A method as in Clause 1F further comprising:

the engine instructing assembly of the first tray from a plurality ofinjection molded parts based upon dimensions of the first product.

Clause 1G. A method comprising:

an engine referencing data in a database to display a plurality ofrelated products, the data comprising a product identifier and a productquality;

the engine receiving from a customer a single selection of a category ofthe related products; in response to the single selection, the engineinstructing a system to dispense a subset of the related productsmatching the category.

Clause 2G. A method as in Clause 1G wherein the plurality of relatedproducts comprise ingredients of a meal recipe.

Clause 3G. A method as in Clause 2G wherein the category comprises ameal type.

Clause 4G. A method as in Clause 3G wherein meal type comprisesbreakfast, lunch, dinner, or dessert.

Clause 5G. A method as in Clause 1G further comprising:

the engine instructing the system to dispense the subset further basedupon a second input removing certain products from the entire set ofrelated products.

Clause 6G. A method as in Clause 1G further comprising:

the engine receiving a second input specifying a substitute product; and

the engine instructing the system to dispense the substitute product inplace of one of the products of the subset.

Clause 7G. A method as in Clause 1G wherein the plurality of relatedproducts are based upon a purchase history of the customer stored in thedatabase.

Clause 8G. A method as in Clause 1G wherein the plurality of relatedproducts are based upon a preference of the customer stored in thedatabase.

Clause 9G. A method as in Clause 1G wherein the plurality of relatedproducts are based upon a budget of the customer stored in the database.

Clause 10G. A method as in Clause 1G wherein the plurality of relatedproducts are based upon an existing inventory of the customer stored inthe database.

Clause 11G. A method as in Clause 1G further comprising:

sorting a plurality of related products based upon a quality assignedaccording to an initial inspection; and

loading the plurality of related products into respective trays basedupon the sorting.

Clause 12G. A method as in Clause 11G further comprising:

the engine instructing assembly of the plurality of trays from injectionmolded parts based upon dimensions of the plurality of related products.

Clause 13G. A method as in Clause 11G wherein the initial inspectioncomprises an optical inspection.

Clause 14G. A method as in Clause 13G wherein the optical inspectioncomprises a visual inspection.

Clause 15G. A method as in Clause 11G wherein the initial inspectioncomprises spectroscopy.

Clause 16G. A method as in Clause 11G further comprising:

the engine instructing storage of the respective trays under controlledenvironmental conditions based upon the inspection.

Clause 17G. A method as in Clause 11G further comprising:

the engine instructing a subsequent inspection prior to dispensing theplurality of related products.

Clause 18G. A method as in Clause 17G where the subsequent inspection isof a same type as the initial inspection.

Clause 19G. A method as in Clause 17G where the subsequent inspection isof a type other than the initial inspection.

Clause 20G. A method as in Clause 20G wherein the initial inspectioncomprises an optical inspection.

Clause 1H. An apparatus comprising:

a plurality of unique individual product items stored upon trays withina plurality of carousels under different environmental conditions;

a non-transitory computer-readable storage medium configured to recordat least a location and a status of each unique individual product item;

a graphic user interface (GUI) configured to offer a customer selectionbetween a first unique individual product item and a second uniqueindividual product item; and

an engine in communication with the GUI and the non-transitorycomputer-readable storage medium, the engine configured to instructdispensing of the first unique individual product item from a firstcarousel into a packaging in response to the customer selection.

Clause 2H. An apparatus as in Clause 1H wherein the first uniqueindividual product item and the second unique individual product itemdiffer from each other based upon an attribute.

Clause 3H. An apparatus as in Clause 2H wherein the attribute is storedin the non-transitory computer readable storage medium associated withthe location and the status.

Clause 4H. An apparatus as in Clause 2H wherein the attribute is basedupon a first inspection performed prior to the dispensing.

Clause 5H. An apparatus as in Clause 4H wherein the first inspection isperformed prior to loading the first unique individual product item intothe first carousel.

Clause 6H. An apparatus as in Clause 5H wherein the first inspection isperformed with the first product item disposed on a tray.

Clause 7H. An apparatus as in Clause 5H wherein the first inspection isperformed prior to the first product item being disposed on a tray.

Clause 8H. An apparatus as in Clause 7H further comprising an apparatusconfigured to assemble the tray from a plurality of injection moldedpieces upon receipt of an instruction from the engine.

Clause 9H. An apparatus as in Clause 4H wherein the first inspectioncomprises rotating the first unique product item to obtain multipleviews.

Clause 10H. An apparatus as in Clause 9H further comprising a conveyorconfigured to handle the first unique individual product item during thefirst inspection.

Clause 11H. An apparatus as in Clause 10H wherein the rotating isorthogonal to a direction of travel along the conveyor.

Clause 12H. An apparatus as in Clause 10H wherein the conveyor comprisesa pivoting belt conveyor.

Clause 13H. An apparatus as in Clause 10H wherein the conveyor comprisesa star wheel.

Clause 14H. An apparatus as in Clause 10H wherein the conveyor comprisesa roller.

Clause 15H. An apparatus as in Clause 14H wherein the conveyor comprisesa spinner and roller conveyor.

Clause 16H. An apparatus as in Clause 10H wherein the conveyor comprisesa pop up conveyor.

Clause 17H. An apparatus as in Clause 4H further comprising a gantry toperform the rotating.

Clause 18H. An apparatus as in Clause 4H further comprising a camera toperform the first inspection.

Clause 19H. An apparatus as in Clause 2H wherein the attribute isfurther based upon a second inspection performed during the dispensing.

Clause 20H. An apparatus as in Clause 2H wherein the attribute comprisesa current quality, a projected future quality, or a ripeness.

Clause 1I. An apparatus comprising:

a plurality of unique individual product items stored within a pluralityof carousels under different environmental conditions;

a non-transitory computer readable storage medium recording,

an inspection result of each of the plurality of unique individualproduct items, and

a location of each of the plurality of unique individual product items;

a processing engine in communication with the plurality of carousels andwith the non-transitory computer readable storage medium; and

a plurality of stations associated respectively with the plurality ofcarousels and configured to dispense unique product items into packagingupon receipt of an instruction from the processing engine.

Clause 2I. An apparatus as in Clause 1I wherein the non-transitorycomputer-readable storage medium further records a Stock Keeping Unit(SKU) related to the inspection result and the location.

Clause 3I. An apparatus as in Clause 1I wherein the non-transitorycomputer-readable storage medium further records a product item qualityrelated to the location and generated from the inspection result by theprocessing engine.

Clause 3I. An apparatus as in Clause 1I wherein the inspection resultcomprises an image.

Clause 4I. An apparatus as in Clause 3I wherein the image comprisevisible wavelengths.

Clause 5I. An apparatus as in Clause 3I wherein the image comprisesother than visible wavelengths.

Clause 6I. An apparatus as in Clause 1I wherein the inspection resultcomprises a spectrograph.

Clause 7I. An apparatus as in Clause 1I wherein the inspection resultcomprises chemical sensing.

Clause 8I. An apparatus as in Clause 7I wherein the chemical sensingdetects spoilage.

Clause 9I. An apparatus as in Clause 7I wherein the chemical sensingdetects a mold spore.

Clause 10I. An apparatus as in Clause 1I wherein sorting the pluralityunique individual product items into the plurality of carousels is basedupon the inspection result.

Clause 11I. An apparatus as in Clause 1I further comprising a mechanismconfigured to handle the plurality of unique individual product itemsduring inspection for disposing on a tray.

Clause 12I. An apparatus as in Clause 11I wherein the inspectioncomprises rotating a unique product item to obtain multiple views.

Clause 13I. An apparatus as in Clause 12I wherein the mechanismcomprises a conveyor.

Clause 14I. An apparatus as in Clause 13I wherein the rotating isorthogonal to a direction of travel along the conveyor.

Clause 15I. An apparatus as in Clause 13I wherein the conveyor comprisesa pivoting belt conveyor.

Clause 16I. An apparatus as in Clause 13I wherein the conveyor comprisesa star wheel.

Clause 17I. An apparatus as in Clause 13I wherein the conveyor comprisesa roller.

Clause 18I. An apparatus as in Clause 13I wherein the conveyor comprisesa spinner and roller conveyor.

Clause 19I. An apparatus as in Clause 13I wherein the conveyor comprisesa pop up conveyor.

Clause 20I. An apparatus as in Clause 12I wherein the mechanism furthercomprises a gantry to perform the rotating.

FIG. 76 offers a simplified block diagram showing an overview of asystem according to an embodiment. Specifically, system 7600 comprises aplatform 7602 including a processing engine 7604 and a graphic userinterface 7606 in communication with a customer 7608.

The platform is in communication with a facility 7610 via acommunications network 7611. In particular, the engine is configured toprovide instructions 7612 to cause the facility to intake 7614 aplurality product items in bulk form, to perform an inspection 7616(e.g., utilizing camera 7617) of those bulk items, and to sort 7618those bulk items on the basis of the inspection.

Further on the basis of the inspection, the platform is furtherconfigured to instruct an apparatus 7620 to assemble trays 7622 fromindividual parts 7624 (e.g., injection molded pieces). The trays thenreceive the sorted product items, and are in turn loaded 7626 intocarousels 7628. As has been described extensively above, those carouselsare maintained under particular set(s) of environmental condition(s).

The platform is further in communication with non-transitorycomputer-readable storage medium 7630 having data stored thereon. Forexample, database 7632 may comprise data regarding various aspects ofthe system and the products stored therein.

Data stored in the database may be referenced by the platform to fulfill7633 a customer order. In particular, the customer may issue to theplatform, a request 7640 specifying a particular item type 7642 (e.g., aspecific apple varietal such as golden delicious). The platform mayreceive the request, and reference that item according to a tablestoring SKU information.

Then, with reference to data in the database, the platform may return todisplay to the customer in the GUI, data regarding a plurality of uniqueindividual product items 7644, 7646, and 7648 matching the request. Thedata may comprise specific information 7650 regarding those individualproduct items, including but not limited to an image, a lifetime, andothers.

Then, the GUI is configured to receive a user selection 7652 of one ofthe particular product items. In response, the engine is configured toinstruct a dispensing station 7653 of the facility to dispense thatunique individual item from the tray and carousel, into packaging 7654supported in a moveable traveler 7656.

While the particular embodiment shown in FIG. 76 depicts the routing ofproduct items directly from the dispensed carousel to a shipping stationfor delivery, this is not required. Alternative embodiments could employan intermediate carrier shuttle structure. There, items are dispensed tothe intermediate carrier shuttle, that is in turn brought to a differentlocation for consolidation of product items into packaging.

In some embodiments, it is noted that the processing engine maycommunicate to the facility, an instruction to transfer 7657 a productitem from one storage carousel to another. Such a transfer instructioncan be based upon considerations such as an expected ripeness of theproduct item, a storage capacity of a particular carousel, and anexpected future order of a product item (e.g., as may be predicted bymachine learning). This transfer can be accomplished utilizing atraveler (as shown), or alternatively utilizing a different systemcomponent.

According to certain embodiments, the product dispensing and/or transfermay be accompanied by an inspection 7680 (e.g., as performed by a camera7682). This dispensing/transfer inspection may be the same, or may bedifferent, from the original inspection forming the basis for theinitial sorting.

Finally, once all of the selected customer items have been collected inthe packaging, the engine is configured to instruct the traveler to movealong path 7658 to shipping station 7660. There, the packaging isoffered for delivery 7662 to the customer.

According to certain embodiments, this delivery may comprise thecustomer receiving the product items in the packaging as conveyed via avehicle (e.g., drone, truck, car, bicycle).

According to alternative embodiments, however, delivery could notfeature an automated drop, but rather pickup by a human customer. Suchmanual delivery options could utilize visual cues such as indicatorlights or screen cartoons.

It is noted that the non-transitory computer readable storage medium ofFIG. 1 further comprises a knowledge base 7634. The data of thatknowledge base may be referenced by the engine in performing machinelearning processes 7690, as has been mentioned previously. Those learnedcharacteristics may be utilized to enhance efficiency of systemoperation and the quality of the customer experience interacting withthe system.

For example, according to some embodiments a customer may be able toprovide feedback about specific items. That feedback can be used tobetter predict preferences and other quality attributes.

Furthermore, information regarding item availability able to be combinedwith expected demand and expected inventory replenishment, could beutilized to incentivize specific consumption trends. This can beaccomplished by, amongst other techniques, placement in web browsers,relative pricing, promotions, and suggestions of recipes that includesuch items or otherwise.

FIGS. 77A-C show front, top, and perspective various views of oneembodiment of a design 7700 for an inspection station. Here, the productitems 7702 enter on the left, move along the conveyor 7703 to thecenter.

The item gets lifted by the rollers, the rollers spin and cameras takepictures from above. FIG. 77D shows a perspective view with the rollers7704 raised.

The items is dropped back onto the conveyors and the items advancesforward with its classification. FIG. 77E shows a perspective with withthe rollers lowered.

FIG. 78A shows a perspective view of an alternative carrier design.According to this embodiment, the carrier 7800 features plastic walls7802 to hold the tray flange in space.

To locate the tray, a click in place feature was added to the carrierand tray. Specifically, when the tray is inserted, it moves past fourclasps 7804 (two on each side of the tray) that are shown in the walledge view of FIG. 78B, and the enlarged view of FIG. 78C. These claspsbend back during installation and then lock the tray in place onceinstalled. This keeps the tray centered in the carrier and it keeps itfrom sliding out or shifting while the carousel rotates.

The carrier embodiment shown may offer benefits by avoiding theconsumption of space otherwise needed to fit the forks into the carrierto remove the tray. Now, without that consumed space, the tabs can besmaller.

Also, because the dispensing tool does not need to fit into the carrier,a higher number of slots can be designed for trays. In this particularembodiment, 2× the number of slots have been added. This affordsflexibility when inserting trays into the carrier, to achieve maximumdensity.

According to some embodiments, an interior of a carousel may be linedwith foam panels. These foam panels attached to the walls with a hookingbracket, illustrated in perspective and side views in FIGS. 79A-B.

Here, the hooking bracket 7900 has teeth 7902 that sink into the foam ofthe panels. The bracket may or may not be glued on, too. Also, thebracket may be covered with some of the same coating that the foam iscovered in. The bracket may or may not be inset into a pocket in thefoam.

The frame of the carousel could have a projecting stud 7904 with whichthe bracket could mate. So, to install the panel, the foam gets liftedso the stud goes through the large holes. Then the panel is lowered downand two or more brackets get secured to the studs as they slide into thekey hole shape.

FIGS. 80A-B show various views of an embodiment of a dispensingapproach. This embodiment 8000 features hooks 8002 that engage the tray8004 from the front of the carrier.

Such an approach may offer certain benefits. For example, it avoidstolerance considerations attributable to the carrier tilting backforward. There will be a tolerance in the location of each carrierrelative to each other.

With this design, the hook fits into a large area and has a bigtolerance itself. Now, the position can be off by >5 mm and still engagecorrectly. This increases tool robustness, and may reduce the cycle timeof the dispensing process because less precision is needed.

FIGS. 81A-C show perspective, side, and top views respectively, of anembodiment of a dispensing approach 8100. Here, the vertical lift designis incorporated into the pop-through conveyors. In place of splines,motor(s) 8102 were put onto each pop through conveyor. This offerspotential savings in cost and complexity, in that stepper motors arecheaper than a single servo plus all the splines and pulleys required todrive them.

This embodiment also features a left-to-right belt drive design 8104that has a fixed timing belt 8106 and pulleys 8108 that move through it.Again, this reduces cost and complexity, disposing the motors onto thepop through conveyor and making the tool easier to manage.

FIG. 82 shows a side view illustrating an embodiment of a dispensingapproach. Here, product items are moved out from the dispensing toolonto a flat table or a set of conveyors that are built into a surface ofthe traveler platform 8200. The traveler than moves to the shipping areaand an order is consolidated.

This approach allows larger-sized items to be stored and dispensed. Suchitems may not normally fit into a bag, but this design saves the manualeffort of walking down an aisle and picking them from a shelf.

FIG. 83 shows a top view illustrating an embodiment of a dispensingapproach. Here, product items are moved out of the dispensing tool. But,instead of the customer bag, the product item drops into a cushionedpocket 8300 that protects the item. The traveler might have multiplebuckets that items can be dropped into. These items are then moved tothe shipping area and consolidated with the larger order.

This could be used when items are too fragile to be dropped onto other,harder items in a bag. A peach, for example, might get damaged fallingonto a box corner or the stem of a butternut squash.

Clause 1J. An apparatus comprising:

a frame translatable to a staging position in front of a carousel;

a fork supported by the frame and configured to extend in a directiontoward the carousel to engage a tray holding an item; and

a moveable member configured to,

project vertically into a first opening of the tray,

contact the item at a first point, and

dispense the item from the tray in a direction away from the carousel,into packaging.

Clause 2J. An apparatus as in clause 1J wherein the moveable membercomprises a conveyor.

Clause 3J. An apparatus as in clause 2J wherein the conveyor comprises:

a first conveyor belt configured to project through the first openingand contact the item at a first point; and

a second conveyor belt configured to project through a second opening inthe tray and contact the item at a second point.

Clause 4J. An apparatus as in clause 3J wherein the first conveyor beltand the second conveyor belt are parallel.

Clause 5J. An apparatus as in clause 2J further comprising a backstopconfigured to prevent an item from falling off a back side of the trayopposite to the staging position.

Clause 6J. An apparatus as in clause 1J further comprising a load cell.

Clause 7J. An apparatus as in clause 1J wherein the moveable member isconfigured to translate in a horizontal direction toward the stagingposition after projecting through the first opening.

Clause 8J. An apparatus as in clause 1J further comprising an itemlocation sensor.

Clause 9J. An apparatus as in clause 8J wherein the item location sensorcomprises an optical sensor.

Clause 10J. An apparatus as in clause 9J further comprising an opticalfiber.

Clause 11J. An apparatus as in clause 1J wherein the moveable member isconfigured to move in a vertical direction to project into the firstopening.

Clause 12J. An apparatus as in clause 1J wherein the tray is configuredto move in a vertical direction to allow the moveable member to projectinto the first opening.

Clause 13J. An apparatus as in clause 1J wherein the tray comprises aninjection molded part.

Clause 14J. An apparatus as in clause 13J wherein the tray comprises aplurality of injection molded parts assembled by a machine.

Clause 15J. An apparatus as in clause 1J wherein the fork comprises ahook configured to engage a hole of the tray.

Clause 16J. An apparatus as in clause 1J wherein the fork is configuredto extend along a side of the tray.

Clause 17J. An apparatus as in clause 1J wherein the fork is configuredto align with the tray utilizing a pin.

Clause 18J. An apparatus as in clause 1J further comprising a pulley.

Clause 19J. An apparatus as in clause 1J further comprising a spline.

Clause 20J. An apparatus as in clause 1J further comprising a belt.

As discussed at length above, embodiments of product handling systemsmay feature traveler(s) that are configured to receive dispensed itemsand to transport same for delivery (e.g., to a customer's vehicle or adelivery vehicle—manned or unmanned). It is noted that a variety ofdifferent mechanisms in the form of a Line Following Robot (LFR) may beutilized by embodiments for product handling purposes.

Examples of such mechanisms are summarized below in the following table.

NAME ACRONYM DESCRIPTION Line Following Bag LFB robot bearing acollapsible bag on top (e.g., for receiving a plurality of productitems) Line Following Mini- LFM robot bearing multiple Bags mini-bags(e.g., for receiving individual product items such as produce, meat,etc.) Line Following Platform LFP robot with a top platform that canreceive and move cases or bulk goods Line Following Box LFBX robot withbox on top designed to interact with dispensing system Line FollowingTray LFT robot that carries trays

Here, the term “Line Following” does not necessarily require guidancethat is based upon line following principles. Rather other forms ofmovement are possible, including but not limited to the use of markers,active radar, and others.

According to certain embodiments, product handling approaches mayfeature item storage on fixed shelving rather than in a moveablecarousel. FIG. 84 shows a simplified cross-sectional view of oneembodiment of a product handling apparatus.

This product handling apparatus 8400 comprises moving carousel 8402.FIG. 84 further shows the LFT 8404, LFB 8406, and dispense station 8408mechanisms as described herein, which are used for product handling.(The letter “D” is used throughout as a shorthand to indicate amechanism for dispensing individual product items from a tray intocorresponding packaging, e.g., a bag, a mini-bag, a box, or a platform).

By contrast, FIG. 85A shows a simplified cross-sectional view of analternative to a moving carousel that features a fixed shelvingapproach. This product handling apparatus 8500 also comprises a LFT8504, a LFB 8506, and a dispense station 8508.

The product handling apparatus of FIG. 85A also comprises a central liftstorage system 8510 including a vertical lift module 8512. The centrallift storage system would transport trays bearing individual productitems from the fixed shelving storage 8514 to the dispensing station andultimately for dispensing into a bag.

In particular, the lift mechanism may comprise a shuttle that can moveup and down to contact and remove a tray from the fixed shelvingstorage. The tray is then brought downward to the dispense station,where product items are dispensed into the LFB.

FIGS. 113A-D show various views of this interaction between a verticallift mechanism accessing fixed shelving for storage, and a dispensemechanism. In particular, FIG. 113A shows a side perspective view of alift 11300 located within a shaft 11302 and in the process of retrievinga tray 11304 from the fixed shelving 11305. Further details regarding anembodiment of a lift are provided below in connection with FIGS. 102A-B.

The view of FIG. 113A further shows a portion of the dispense station11306. In particular, only one of the two independently positionableconveyors 11308 is depicted, as has previously been described inconnection with at least FIGS. 20-21, 43A, and 45B.

FIG. 113B shows the lift holding the tray descending over the conveyorsof the dispense station. FIG. 113C shows the conveyors independentlymoving to align to the width of the slots in the tray.

FIG. 113D shows the elevator continuing to descend. As a result, theconveyors project through the slots in bottom of the tray. This liftsoff of the tray, the items housed in that line of the tray. Movement ofthe conveyors can then result in the dispensing of the first and thensubsequent items in the line.

FIG. 85B shows a simplified cross-sectional view of a variation of theproduct handling apparatus of FIG. 85A. Specifically, for the producthandling apparatus 8520, no separate, dedicated lane is provided for theLFT, which must access the storage utilizing the same available space asthe LFBs. While increasing storage density, this alternative embodimentmay reduce the flexibility by which incoming stocked trays are placedinto the fixed shelving.

Returning to FIG. 85A, as described above this shows the dispensestation as being located directly underneath the shuttle that is movingvertically up and lowered with the tray. This configuration avoids theneed for the conveyors of the dispense station to have the ability tomove vertically in order to project through the tray slots (the liftaccomplishes this vertical movement instead).

However, this is not required and alternative embodiments may employ adispensing mechanism that is itself vertically moveable, in conjunctionwith products that are supported on fixed shelving rather than in amoveable carousel. FIG. 85C is a simplified cross-sectional view of suchan embodiment of a product handing apparatus 8530, with the dispensingmechanism 8532 being vertically moveable as shown.

FIG. 85D shows a simplified cross-sectional view of a variation of theproduct handling apparatus of FIG. 85C. Specifically, for the producthandling apparatus 8540, no separate, dedicated lane is provided for theLFT, which must access the storage utilizing the same available space asthe LFBs. Again, while increasing storage density, this alternativeembodiment may reduce the flexibility by which incoming stocked traysare placed into the fixed shelving.

Returning to FIG. 85A, that shows a configuration where the LFT isdisposed to access the fixed shelving storage from a same side of thedispensing station as the LFBs. However, this is not required.

As shown in FIG. 85E, an embodiments of a product handing system 8550could position the LFT on the opposite side of storage from the dispensestation. This opens up space to accommodate an additional LFB.

Moreover the further alternative embodiment 8560 shown in FIG. 85Freveals that this additional space could instead be condensed. Thiswould in turn increase storage density.

While the embodiment of FIG. 85A shows as single vertical shuttle pershaft, this is not required. Alternative embodiments can employ multiplelift shuttles.

FIG. 86 shows a simplified cross-sectional view of a design having oneor more lift mechanisms located in a same lift shaft. Such an approachcan be utilized to increase the speed of the tray picking process. Whileone shuttle is at the bottom dispensing an item, another shuttle can beabove picking a tray.

The left portion 8600 of FIG. 86 shows a dispense-under-lift approach.Here, one lift 8602 performs tray fetch and replaces in serial. One lift8604 picks from area 8606 and moves to dispense. One lift 8608 dispensesfrom upper area 8610 and moves to area 8612.

The right portion 8620 of FIG. 86 shows a dispense-next-to-liftapproach. Here, lift 8622 replaces used trays. Lift 8624 fetches newtrays.

Lift 8626 replaces used trays. Lift 8628 fetches new trays. Tray 8630brings trays from deep storage into the lower area.

To reduce cost, it may be desirable for a product handling system tomaximize storage density. Environmental control (e.g., temperature,humidity, gas species and concentration control, particulate filtering)in such a product handling system allows significant shelf lifeextension for perishable items. However, environmental control may callfor sealing and/or insulating storage, reducing density.

Accordingly, embodiments may integrate sealing, insulation, and ductingto structural members. This allows for the benefit of environmentalcontrols with little detriment to storage density.

FIG. 87A shows a top view of a repeating shelving design 8700 comprisinginsulated zones 8702, non-insulated zones 8704, and elevator shafts8706. This example layout shows the freedom to distribute environmentalzones. This environmental control design is modular in form and allowsfor climate controlled and non-climate controlled units to resideanywhere in the grid.

FIG. 87B shows a perspective view of one unit 8710 in a repeatingshelving design. Insulation panels 8712 reside in interior of structuralmembers

Perforated duct plate 8714 turns structural members into a designatedrefrigeration air path. The air exits holes 8716 and refrigerates itemson the shelving slots

FIG. 87C shows an enlarged top view. Air flows through cavity 8730 asshown by the arrow.

Under certain circumstances, a product item may need to be retrievedfrom a center of a tray. Such a center-tray-located item can bedelivered by adding a second set of conveyors behind the dispense area.This provides a low cost way to dispense an item from any location inthe tray.

FIGS. 88A-88F show simplified side elevational views of a central liftstorage system configuration allowing center tray picking according toan embodiment, in operation. FIGS. 89A-89F show corresponding top viewsof retrieving a particular item (here a T-bone steak) from the center ofa tray.

FIGS. 88A and 89A show a first step. Here a full tray 8800 arrives atthe dispensing area 8801, and the customer requests a specific item 8802(here a particular cut of meat) presently located at the tray center.

FIGS. 88B and 89B show a second step. The LFB 8810 bearing the bag 8808arrives. The back conveyor 8804 matches the row with the desired item.The dispense conveyor 8806 lifts the row of items and conveys backward.The back conveyors convey simultaneously until the item required islocated on the back conveyor.

FIGS. 88C and 89C show a third step. Here, the tray lifts up from thedispense tool while the items on the back conveyor stay in place.

FIGS. 88D and 89D show a fourth step. Both conveyors 8804 and 8806convey forward until the sought-after item is on the dispense conveyor.

FIGS. 88E and 89E show a fifth step. Here, the back conveyor stopsmoving and the dispense conveyor conveys forward till it dispenses theitem into the bag 8808.

FIGS. 88F and 89F show a sixth step. The tray returns to the dispensetool, and any unused items move back onto the tray. The LFB bearing thebag containing the dispensed item, moves to its next location.

While these particular figures show a specific embodiment that engagesin product handling to access center tray items within the dispensearea, this is not required. Alternative embodiments featuring thedispense conveyor located on the lift, could perform product handlingabove the dispense area somewhere in the storage area. A secondaryconveyor could be in the back or front of the lift.

Embodiments as just described can be utilized when selecting specificcuts of meat, cheese, produce, etc. Such embodiments can also beemployed where multiple, low sale SKUs (like bottles of wine or spices)are present in one lane of the tray. Such embodiments can be used as amethod of combining inventory on less than full rows and trays.

FIG. 102A shows a front perspective view of an elevator design which maybe used with a vertical lift configuration as has been described herein.FIG. 102B shows a side view of the elevator of FIG. 102A.

In particular, motors for the elevator and tray retrieval are locatednear the ground. Power is transmitted to the elevator carriage viatiming belts. This configuration keeps the motors and electronics awayfrom the potentially wet environment (e.g., condensation) of thevertical lift system.

It is noted that certain tray designs may lend themselves for beneficialuse in connection with a vertical lift system. FIG. 103A shows a topview of an embodiment of such a tray design.

FIG. 103B shows an enlarged perspective view of the tray design of FIG.103A. In particular, this tray embodiment features pulling features inthe form of hooks on the front.

FIG. 103C shows an enlarged underside perspective view of the tray ofFIGS. 103A-B being engaged. An extended chain pin engages that hookpulling feature as it comes around the sprocket.

Software may be utilized to implement certain features in producthandling. For example, software may optimize a speed of fulfilment of anorder.

In the case of inventory management considerations, the most commonlypurchased items may be stored in a few adjacent carriers within eachcarousel, or at proximate locations within the fixed shelving. Rareritems may be stored in the remaining carriers or fixed shelving.

Most of the time, the carousel will likely be moving between these few“high runner” carriers. Ultimately, this arrangement will reduce thetime it takes to fill each order.

In the high runner carriers, only a limited quantity of the high runneritems may be stored. Others items are located on less frequently usedcarriers, accessed if the high runner carriers are empty or if thecarousel happens to be closer to that carrier.

Where fixed shelving is used, high runner storage may be located atlower locations relative to the dispense mechanism. This can reduce thetime required for repeated vertical movement in order to access the highrunner storage locations.

There can be empty tray slots in carriers within each carousel.Accordingly, after a dispense routine, a tray can be returned to adifferent location than the location it was originally taken from. Thisimparts flexibility in optimizing fulfilment speed. A routing managercan optimize each LFB route to minimize interference with other LFBs andreduce fulfilment time.

LFBs have the ability to easily pass by other active LFBs. In this way,LFBs can pick items from carousels and bays that are not busy.

A routing manager knows the status of each carousel and bay. The managercan implement planning to minimize time that LFBs wait in line behindother LFBs.

Where possible, each SKU is distributed throughout the store in multiplelocations. SKUs can be distributed in one or more of multiple:

carousels throughout the store,

bays in each carousel,

carriers in each bay, and

trays in each carrier.

This imparts flexibility in optimization by the routing manager. Largerorders can be prioritized over smaller order in order to ensure ordersare fulfilled in an acceptable time.

The routing manager can signal to a carousel and/or bay to prepare atray for dispensing before an LFB even arrives at the bay. This canfurther reduce wait times.

These considerations allow for faster fulfilment time estimates beingoffered to customers when they place an order.

It is noted that carousels can be designed with different numbers ofbays. For a typical carousel with three bays, the actions of all threebays may be coordinated to the movement of one carousel motor. This canresult in unwanted interference and wait times between the bays.However, carousels designed to control only a single bay avoid thisproblem, thus reducing interference and fulfilment times.

Idle LFBs can be stationed throughout the facility in order to reducetravel time to a first fulfilment bay. Carousels can be arranged onpaths of different lengths and shapes to optimize speed of LFBs throughfacility. LFBs can travel in a variety of directions and paths to bestnavigate the facility based on the items to be retrieved from thecarousels.

Once an order is finished, the LFB will travel directly to a waitingdriver in the delivery area. The driver will remove the bags from eachLFB and put them in their vehicle.

This reduces time and labor. There is no need to stop the LFB somewhere,to remove the bag, and then to move it again out to the vehicle.

Where a vehicle has not yet arrived at the delivery area, the LFBs candrive into a temperature controlled chamber until the content of theirbags is ready to be picked up.

Once an LFB is filled, it can drive to a parking lot or delivery area ofthe building to meet a vehicle. The bag can then be removed from the LFBand placed in the vehicle.

In certain embodiments, a human may remove the bag from the linefollowing robot (LFR). In some embodiments, a tool may put the bag intoa (self-driving) vehicle or drone.

Such delivery to a vehicle is not limited to a bag of a LFB. Alternativeembodiments could have the delivered item in a box (LFBX) or sitting ona platform (LFP). Such a procedure may conserve labor as no human beingis required to prepare the order.

The LFR can wait in a queue for the vehicle to arrive. If item(s) needto be refrigerated or to remain frozen, the LFR can move to a cooler orfreezer while waiting for the vehicle to arrive.

Embodiments may split up individual orders amongst multiple LFBs. Eachsuch LFB may then navigate independently throughout the store, receivingitems in parallel.

Dynamic pricing and/or incentives can be used to reduce traffic duringpeak hours. This can more evenly distribute orders over the course of aday.

Software may also be employed to enhance a product inspection processfor quality. Certain embodiments may employ software for qualityinspection in connection with:

tuning a defect detection model to match expert human grading orconsumer preference;

controlling a (3D) camera for defect detection; and/or

implementing a Pass/Fail alternative.

As previously disclosed, machine learning models may be trained todetect defects on visual and NIR images of produce. One approach may bebased upon object detection varieties of machine learning.

However, embodiments are not limited to this or any other type ofmachine learning approach. Alternatives which may be used can includebut are not limited to, anomaly detection (auto-encoders or GANs) orsegmentation techniques.

Prediction models can be developed per individual product SKU.Alternatively, prediction models can encompass multiple types of SKUs.Accordingly a single prediction model can be used to predict quality forten different apple varieties, or a single prediction model can be usedfor all stone fruits.

Three-dimensional images of produce, or three-dimensionalreconstructions of produce from a series of two-dimensional images,could also be used as the basis for training predictive models anddetecting defects. Such an approach can add a variety of benefits,including the ability to judge defect depth and topography.

Machine learning methods according to embodiments can reveal multipletypes of information about each detected defect. Such defect informationtypes can include:

defect type,

defect size, and/or

confidence in prediction.

According to particular embodiments, a second machine learning model cantake some or all of these inputs (e.g., quantity, type, size,confidence) and infer a resulting output grade for the overall producequality. Possible grades may be A/B/C, for example.

This machine learning model can be trained based on a human qualityexpert applying a grade to each item. This machine learning model canalso be trained based on consumer feedback to match consumer preference.

According to embodiments, machine learning models can also be used topredict a simple pass/fail quality grade on items, without firstidentifying specific individual defects. In such an implementation, themachine learning model can be trained on a human quality expert applyinga pass/fail grade to each item.

Alternatively, or in conjunction with human training, the machinelearning model can be trained on sets of images that are individuallylabeled as pass/fail. Thus simple pass/fail grading could also bereplaced by multiple levels of categorization (A/B/C, etc.)

Such approaches could also be extended to items in the store beyondspoilable produce. For example, embodiments could identify damaged cans,boxes, etc. utilizing machine learning prior to storage and/or deliveryto a customer.

Software may also be employed to perform inspection for Quality Analysis(QA) and/or presentation to customer. Such inspection may take place atinduction, with 2D or 3D rendering allowing a user to select theirpreferred item. Such inspection may also occur during fulfilment, forpurposes of Quality Control (QC).

According to certain embodiments, inspection may occur at the time ofinduction of the product to the store. Inspecting (with 2D or 3Dimaging) products such as produce as it enters the store, followed bytracking of items in inventory, customers can be afforded an option toselect specific produce items (e.g. for a cart) based on the savedmages. A series of 2D images can ultimately be used to create a 3Dreconstruction of the produce.

The results of later (after intake) inspection of produce or otherproduct items, can be used to update the 2D or 3D representation(s) thatare offered to customers during the selection process. These imagescould also be used to update a customer after checkout, regarding theitems that have been dispensed into their grocery bags.

Software may also be employed in connection with product inspectionduring fulfilment. For example, cameras attached to dispense stations orthe carousel, can image:

produce as it is being dispensed into the LFBs, or

the bag after each dispense routine has finished.

According to some embodiments, a final image of each bag can be takenafter the LFB has finished collecting all of its items, or aftermultiple bags from different LFBs are combined. These images can be sentto fulfil QA teams for monitoring, or to customers to provide feedbackon the quality of their order during fulfilment.

As previously disclosed, embodiments may utilize software for purposesof produce ripeness prediction. This may involve predicting fruitripeness stage including the use of tools, and optimizing inventory viastorage control.

Environmentally controlled storage vessels (e.g., fixed, or moveablecarousels) allow maintaining product items at different conditions(e.g., temperatures, levels of humidity and ethylene) in order tocontrol their ripeness progression.

Fruits ripen faster when exposed to higher temperatures and ethylene. Bycontrast, colder temperatures generally slow down the ripening process.

LFTs can be used to shuttle ripening product items such as fruits,between different storage environments based upon conditions optimal forthe fruit, in order to maintain optimal inventory for customer needs. Inthis manner, product handing approaches according to embodiments canprovide fruits that are currently ripe, or will be ripe at a predictedfuture time range.

As one possible example, avocados can be kept at about 16° C. until theyreach a ripe state. Then, they can be moved to 3° C. At this coldertemperature, the ripe avocados can be expected to remain ripe for aboutseven days.

Predictive modeling (e.g., accomplished by machine learning or othermethods) can be employed to predict a future progression of ripening ofproduct items. Such models can ingest information from one or moresources including but not limited to:

harvest date,

harvest conditions,

cold storage history,

farm source,

distributor source,

packaging and handling methods,

seasonality,

type of fruit,

variety of fruit, and

measurements of the current state of ripeness.

Using automated, semi-automated, or manual tools for measuring aripeness stage of a fruit or other product can be beneficial inproviding a current state fruit ripeness in a manner that is accurate,repeatable, and predictable.

A variety of measurement methods, employed alone or in combination, mayallow for assessment of a current ripeness stage of a product item. Suchmeasurements can include but are not limited to:

firmness measurements (e.g. with a durometer type tool),

acoustic property measurements (e.g. with a speaker+microphone typetool)

light reflection/interactance/absorption measurements (e.g. with aspectrometer tool).

Such measurements can replace or supplement manual human measurements,including but not limited to:

vision,

smell, and

a hand-score of firmness

In certain cases, periodic (e.g. once a day) measurement of productitems is important for tracking the ripeness stage over time. In othercases, metadata of the product item, along with typical trends for thatitem, may be sufficient for predicting the future ripeness trends.There, a single measurement of ripeness during induction to the storelocation, or even no measurement at all, may be sufficient.

Sourcing items that are pre-conditioned (e.g., ethylene and/ortemperature conditioned) can create more homogeneous batches of fruits.This can allow for sub-sampling when measuring fruit ripeness, or permitfewer periodic measurements over time.

It is noted that tools for ripeness measurement (e.g., firmness,acoustic, spectrometer, etc.) can be located in a variety of differentstore locations to provide periodic assessments of product ripeness.

One possible position within the store is in a temperature controlledinduction or pre-storage location for manual or semi-automatedmeasurements. Ripeness measurement could be integrated into theautomated inspection equipment for measurement during induction. Or,measurement could be performed when LFTs remove the produce fromcontrolled climate storage and return to the station for periodicmeasurements.

Ripeness measurement may be incorporated into the LFT itself, or may bepresent at measurement stations distributed throughout the store foreasy and quick LFT access. Ripeness measurement may be incorporated intodispense stations, or within a storage (e.g., carousel) bay itself.

In an alternative design to a moving carousel, product items could bestored in environmentally controlled individual vertical storage bays.Such storage does not move like a carousel. Rather, trays are removedand returned to fixed locations utilizing a tray transport mechanism.Such a configuration is shown in the simplified cross-section of FIG.90.

The storage temperature can be modular even within a single bay. Thisallows for higher and lower temperatures for granular control overripeness stage. And, such an approach allows for easy transport ofripening produce between different temperature conditions based uponripeness stage, without the need for a LFT to move the trays.

Ripeness measurement tools can be incorporated into this same verticalbay or dispense station for easy periodic measurements of fruits overtime.

A variety of different designs of mechanical apparatuses for producthandling, may be used to accomplish product inspection. Some embodimentsmay use tray loading shuttle conveyors for mechanical inspection design,as shown in FIGS. 91A-B.

In particular, items 9100 can be loaded from an infeed conveyor 9101into multiple trays 9102 using multiple shuttle conveyors 9104 andpop-through conveyors 9105 to move the items to the correct traylocation. There can be one or more shuttles, which can move to one ormore trays.

A mechanism similar to that used for dispensing items from trays, may beused for loading of items onto trays. One example is a dual-beltconveyor mechanism.

FIG. 104A shows a perspective view of an embodiment of tray loadingshuttle mechanism 10400. FIG. 104B shows an overhead view of the trayloading shuttle conveyors of FIG. 104A.

The trays do not move. Once intake inspection of an incoming item isconcluded, the infeed conveyors move that item forward onto the shuttle.Then, the shuttle moves the item laterally until it is aligned with theproper row of the tray upon which the item is to be loaded.

Particular embodiment of designs for mechanical inspection may employrollers with spirals that turn an item ninety degrees. FIG. 92 shows asimplified top view of such a design for mechanical inspection inoperation.

In particular, such rollers may handle both spherical and cylindricalobjects. Spinning spiral roller(s) can rotate non-spherical objects totheir equilibrium position. Spiral ridges on the rollers contact objectsdiagonally with respect to normal.

Objects can shift lengthwise along path of the conveyor.Roller-to-roller distance may be adjustable to account for differentobject size.

FIGS. 105A-B show overhead views of inspection size detection adjustmentin operation, according to an embodiment. In particular, average SKUsize may be predetermined so that rollers will adjust to a best knownsize position.

Size detection camera(s) can recognize outliers to determine whetherrollers need to be adjusted to account for large variances in size.Asymmetric objects can also be detected and accounted for by positioningrollers relative to predicted center of gravity. The roller assembly canmove along a rail in order to adjust for size.

One example of an embodiment of an inspection induction system is shownin FIG. 93. Various forms of produce that are to be inspected, proceedthrough a series of conveyors. There, the produce is automaticallysingulated and aligned after being placed onto an unload conveyor.

Loose produce corresponds to individual spherical and cylindricalproduce. Packed produce corresponds to clamshells, boxes, and bags ofmultiple produce (e.g., lettuce, bags of carrots, etc.)

Some embodiments of mechanical designs may feature circular V beltloading stations. Operation of such a design for loading is shown in thesimplified side view of FIG. 94.

In particular, this station is a high density loading station for bothproduce and dry goods. It allows a buffer to be loaded with items priorto entering either the inspection station and/or tray loading station.

One section of the conveyor belt is loaded up. Then, the station rolls,presenting a free conveyor section to an operator and a loaded sectionof conveyor to the conveyor leading to the next station. The conveyorscan angle up to be a V for spherical and cylindrical items and flattenfor packed items.

This buffer allows the operator to load the station, and then leave toaccomplish another task. FIG. 94B is a simplified view showingunloading.

Some embodiments of product handling mechanisms may feature atranslating shuttle buffer. This offers a high density loading stationfor both produce and dry goods which allows a buffer to be loaded withitems prior to entering either the inspection station and/or trayloading station.

As shown in the simplified view of FIG. 95A, one section of the conveyorbelt is loaded up, and the station translates. This presents a freeconveyor section to an operator, and also presents a loaded section ofconveyor for unloading to the conveyor leading to the next station(shown in the simplified view of FIG. 95B).

The conveyors can angle to assume a V-shape for handling spherical andcylindrical items, or flatten to handle for packed items. This buffermechanism allows the operator to load the station and walk away toaccomplish another task in the warehouse.

Some embodiments of product handling systems may utilized a stackedshuttle buffer. This station is a high density loading station for bothproduce and dry goods which allows a buffer to be loaded with itemsprior to entering either the inspection station and/or tray loadingstation.

As shown in the simplified side view of FIG. 96A, one section of theconveyor belt is loaded up, and then the station translates. Thispresents a free conveyor section to an operator, and also presents aloaded section of conveyor to the conveyor leading to the next station(shown in the simplified side view of FIG. 96B.

Again, the conveyors can angle up to be a V for spherical andcylindrical items, and flatten for packed items. This buffer allows theoperator to load the station and walk away to accomplish another task inthe warehouse

Certain mechanical designs may allow for dynamic storage of cases ofproduct. In particular, for a streamlined product handling system, casesof produce may be stored in a manner promoting autonomous dispensing andan ability to be brought directly to the tray loading station.

As shown in the simplified top view of FIG. 97A, pallets are broken downand individual cases loaded onto an infeed conveyor 9700. These productcases are transferred to the 2D gantry conveyor 9702.

The product cases are then transferred into a bay in the shelving units9704. As shown in the simplified cross-section of FIG. 97B, whensummoned the cases are transferred back onto the 2D gantry conveyor andtaken to the outbound conveyor 9706.

The outbound conveyor carries the case to the tray loading station. Theshelving is a conveyor that is driven by a connection to the gantryconveyor. In particular, geared connection between the motorized gantryconveyor and a non-motorized shelving unit allows transfer of casesbetween the two units

Embodiments may provide the functionality of permitting automaticre-inspecting. That is some SKUs may benefit from a second or thirdinspection on a specific schedule. Thus a LFT can be routed to remove atray from storage, bring the tray to the inspection station, and offloadthose SKUs back into the inspection process. Those re-inspected itemscan then be resorted onto new trays and replaced in the storage.

Alternatively, a camera above each pick station can perform inspectionaccording to a schedule. If unacceptable items are found, they can beremoved from the trays. If the ripeness or quality status has changed,the database of information can be updated accordingly.

Embodiments may offer the ability to automatically combine inventory.Specifically, throughout the fulfilment process each tray will haveitems dispensed from it for fulfillment to the customer.

Eventually, this repeated dispensing will result in trays being onlypartially filled. Such a condition may be undesirable because totalinventory count is reduced.

At least two possible approaches may be used to address this issue. Oneapproach is recombination.

Specifically at low demand times a recombination schedule may be createdrouting LFTs to each partially-filled tray. The trays will be removedand returned to the tray filling station. Trays will be removed from theLFT and placed on the loading station. Using the tray loading shuttleconveyors, for example, items can be removed from some trays and addedto others until some trays are full and others are empty. At this point,the full trays can be replaced in the carousel.

Another approach involves addition. This proceeds similar torecombination, but rather than using the tray loading shuttle conveyorsto recombine items, newly inspected items can be added to the partiallyfilled trays.

Various features of tray designs are now described. It is noted thatsome product items do not interact as well with the dispense station.This can be due to one or more of:

the odd shape of the product,

the presence of many leafy or delicate features,

the presence of a center of mass in a non-advantageous location.

A method allows such product SKUs to be dispensed in a similar manner toother items in the store. Specifically, each items gets placed on asmall raft that fits with the tray. When the item is called for, theitem and the raft are dispensed together.

In one embodiment the raft is captured before moving into the LFB. In analternative embodiment the raft moves into the LFB and is removed at theend by a worker or a tool. In still another embodiment the raft isshipped to the customer for disposal.

Certain embodiments may utilize a tray design that allows dual sidepicking. This is shown in the simplified top view of FIG. 98A, and inthe enlarged perspective view of FIG. 98B.

In particular, the tray includes pulling features that interface withthe arms of the dispense station in order to contact and place thetrays. The symmetrical design of the tray allows the tray to be used ineither orientation, and allows the tray to be contacted and placed ineither direction. This is discussed later below in connection with atleast FIGS. 101A-F.

Certain specialized tray designs may be employed for handling of heavy,bulk items. rays for heavy, bulk items. Examples can include but are notlimited to Examples to 24-packs of bottled water, toilet paper, papertowels, bales of firewood, 24 packs of beer, cases of wine, and boxes ofproduct that have not been unpacked.

FIG. 106 shows a perspective view of an embodiment of a tray designconfigured to handle such items. This tray would store items that do notget dispensed into a bag. Items would sit on this special tray that cansupport a larger weight. The tray would have two or more slots toaccommodate the pop-through conveyors contacting and picking up theitem.

Items would sit on this tray platform and be lifted off, conveyedforward, and then conveyed onto a Line-Following Platform (LFP). The LFPwould then transport the item to the shipping area. Details regardingLFP embodiments are discussed further below.

Returning now to product storage within a moveable carousel, embodimentsthereof can include various features. One such feature is a quickinstall carrier attachment.

Specifically, the storage carousel operates by moving shelving units.Mounting such shelving units to the components inside the carouselduring installation, may be made more difficult by factors such as:

space constriction,

a need to align the carriers angularly (theta) and side-to-side (x), and

high torsional strength required of the joint.

Accordingly, FIGS. 99A-C show perspective, side, and cross-sectionalviews of an attachment design according to an embodiment. Thisattachment design allows for installation with only inside access, twodegrees of freedom for adjustment/alignment, and very high holdingstrengths due to the wedged design.

As shown in FIG. 99B, the attachment method slides in X and theta on theshaft when the circular bolt pattern is loose. Torquing the boltsresults in rigid coupling of the shaft to a side sheet on a carrier.

Embodiments may use a wedge pushed into a cone by screws to generateradial clamping force on the shaft. This fixes the joint radially andaxially. The large mechanical advantage from screw+wedge, allows largeclamping forces to be generated.

It is noted that certain carousel designs may include a single bay.Carousels can include various numbers of bays, all controlled by asingle motor. Typical options include 1, 2, 3, 4 or 5 bays.

The more bays per motor, the larger the cost savings. This is becauseeach bay shares the mechanics required for movement.

However, the fewer bays per motor, the faster the system can operate.

Each bay may be tied to an independently operating bay on the pickstation. When more than one pick station bay is linked with a multi-baycarousel, conflicts may arise slowing the system and creatingbottlenecks.

Embodiments may vary the number of different sized carousels in afacility in order to improve the fulfilment process. Thus for oneparticular facility it might make sense to have all single baycarousels. However, another facility may best utilize a combination ofdifferent sizes. This determination can balance speed requirements withcost.

Various features of a dispensing station according to certainembodiments, are now disclosed. In particular, FIG. 100A shows aperspective view of a dispense station 10000. In particular, thisdispense station adopts a dispense-next-to-lift approach (e.g., as isshown in the right hand side of FIG. 86).

FIG. 100B shows an enlarged view of the dispense station. The dispensestation comprises a separate tray picking mechanism 10002 and a separateproduct dispensing mechanism 10004.

FIG. 100C shows an enlarged view of the tray picking mechanism. Thiscomprises a moveable platform 10006 for supporting the tray, and abi-directional hook feature 10008 for engaging the tray and pulling itonto the platform. Further details regarding this tray engagementfeature are discussed below in connection with FIGS. 101A-F.

As indicated above, this dispense station embodiment adopts adispense-next-to-lift approach. That is, the lift is not also leveragedto provide the vertical movement allowing the product conveyors toproject through the tray slots.

Hence, the product dispensing mechanism is capable of vertical movementbetween a lowered and raised position. FIG. 100D shows an enlarged viewof the product dispensing mechanism in a lowered position.

FIG. 100E shows an enlarged view of the product dispensing mechanism ina raised position. According to this dispense-next-to-lift approach, thelifting axis is altered from lowering the entire tray over the dispenseconveyors, to instead lifting the dispense conveyors through thefixed-in-place tray.

As has been emphasized throughout, the dual conveyors of the productdispensing mechanism are independently actuable to move together orapart depending upon item size. That is, the conveyors can spreadopen/close depending upon item width.

According to this particular embodiment, FIG. 100F shows the independentactuation 10050 as being accomplished utilizing a motor 10052 that isanchored to the moving carriage that includes the conveyor belt (themotor of the second carriage is not visible in this view). Timing belts10054 are single length fixed in place, and the motor moves the carriagealong the belt.

This is not required however, and alternative embodiments may employother approaches for independently actuating dual conveyors fordispensing of product items. Specifically, FIG. 100G shows an overheadperspective view of an alternative embodiment of a product dispensingmechanism.

Here, motors 10060, 10062 are instead anchored to a fixed frame 10064rather than to the carriages themselves. The first motor 10060 drivestiming belt 10068 to actuate 10069 the carriage 10070.

The second motor 10062 drives timing belt 10072 to actuate 10073 thecarriage 10072. This approach featuring a moving timing belt, waspreviously illustrated in the side view offered by FIG. 21.

As mentioned earlier, a dispense mechanism may feature a hook designthat allows for dual side dispensing (that is, dispensing from each ofopposite sides). FIG. 101A shows an overhead view of a dispensemechanism positioned between a carousel and a plurality of tray rovers.

The carousel and tray rover may not require complicated mechanisms.Rather, they may comprise shelving to hold a stored and dispensed item,respectively.

The arm of the dispensing station can reach in two directions. A firstdirection contacts and places from the carousel. This is shown in theperspective view of FIG. 101B.

A second, opposite direction contacts and places from the tray rover.This is shown in the perspective view of FIG. 101C.

FIGS. 101D-F show enlarged views of details of the bidirectional arm andhook. The hook is split so that the end can dip under the pullingfeature on the tray and then come up behind it.

While FIGS. 101D-F show this mechanism working in the direction towardthe tray rover, the mechanism works from either direction. That is, thesplit hook could move in the opposite direction toward the carousel.

It is noted that a combination color and depth cameras, may take 3Dscans of a bag or box as it is filled with items. Fiducial markers(recognized by color cameras) may be located on a line-following robot,in order to allow transformation of the bag/box coordinate frame to thedispense robot coordinate frame. Using more markers than the minimumnecessary, can increase accuracy and allow transformation in the eventthat some markers are obscured.

According to embodiments, a program may interpret a 3D scan before eachdispense. The program may comprise hardware, software (e.g.,instructions present in a non-transitory computer readable storagemedium), or a combination thereof, executed by a processor.

The program may then select a dispense location based upon one or moreof the following factors:

volume of vertical space available;

drop distance until impact;

damaging features like corners or edges;

relative fragility of dispensed item;

relative fragility of other items already in the bag.

The program may also adjust the platform height of the line-followingrobot, in order to minimize a drop distance, while still ensuringsuccessful dispensing while retaining items already present in the bag.

After each dispensing, the program may interpret another 3D scan,logging the location of new items. A 3D map of item locations evolves,allowing the association of various regions with features such as:

material hardness,

sensitivity to damage,

temperature,

others.

This map of features allows intelligent placement of new items into thebag (e.g., to avoid dispensing soft peaches onto hard cans). Thepost-dispense software program may also predict a dispensing locationfor the next item, thereby priming the line-following robot and the nextdispense robot to meet more quickly at the next dispense location.

FIGS. 107A-B show overhead images of examples of packing a bag withstereo depth vision. In particular, FIG. 107A shows a color image, andFIG. 107B shows a color+depth image of a same dispense sequence.

Fiducials located around a rim of the robot are recognized. The“droppable” area is shown as reference 10700. A dispense location 10702is precisely selected based on a depth scan and existing data about thelocations of items in the bag. The LFR moves horizontally to positionthe dispense location in front of the conveyor belt pair of thedispensing station.

Certain embodiments may follow specialized procedures for dispensingproduct items into a bag. Such approaches can be employed to pack bagswith a degree of intelligence and care comparable to human effort.

According to some embodiments of procedures, packing density can bemaximized, fitting more items into the same space. Damage to items maybe minimized or eliminated, with items packed safely in the traveler sothat they do not fall to the ground.

Some embodiments of packing procedures may rely upon physics simulation.Three-dimensional scans can be utilized in simulations that model thephysical behavior of items falling off of the dispense conveyors andinteracting with items already in the bag. Optimal drop impact andsettling outcomes can be selected.

Particular embodiments of packing procedures may utilize reinforcementlearning. A reinforcement learning model can be implemented usingfeedback on decisions made by the drop program. Such feedback caninclude but is not limited to:

3D scans providing feedback on dispense success (items making it intothe bag/box/mini-bag/platform or not);

item damage reported by delivery personnel;

item damage reported by customers.

Feedback could allow the model to improve success rate by adjustingcertain numerical control parameters. Example parameters can include butare not limited to:

thresholds;

sensitivities;

function constants;

decision/priority assignments.

Certain embodiments may utilize a 3D camera for item height detection.Specifically, when a particular tray is stocked, the heights of items ofthat tray are known.

However in some instances items on the tray may have shifted. In otherinstances, the items could have variable sizes so that the maximumheight is not known.

As has been discussed, a tray is loaded into storage (e.g., fixed, ormoveable carousel). Each position within that storage may have a limitedallowable height. Thus it may be beneficial to select a storage locationaccommodating the maximum height of items on a particular tray, so thatstorage density is optimized. Embodiments may measure the height of allitems, determine the maximum height, and then select a storage locationthat is only slightly larger than that maximum height.

When a tray is first created, embodiments may employ a (stereoscopic)camera to create a heat map of the height of items in the tray. Themaximum height can then be detected from the data. This information isused to select the location to which the tray will travel in the storage(e.g., LFT).

Once the tray is located at the picking station, a (stereoscopic) cameracan be used to check the heights again, verifying that tray items didnot shift during transport. This data can them be used to confirm thefinal location of the tray in the storage.

Each time a tray is removed from storage, there is some risk that themaximum height has changed. This can be rechecked each time, in order toensure that the tray and the items it bears, will still fit.

If a camera or other sensed information reveals that the tray and itemsno longer fits into the storage, one or more of the following actionscan be taken, as follows.

A new location can be selected.

The picking station conveyors can pop up and adjust the item thatshifted.

A new LFB can be deployed and the shifted item can be removed from thetray.

Various embodiments may employ a 3D camera to monitor fulfilmentsuccess. In particular, certain embodiments may fix a (stereoscopic)camera above the picking station.

That camera can monitor the low of items in and out of the bag. Thecamera can be used to determine the location of items in the tray.

This location information can be fed back to the control software, sothat the correct number of items is dispensed. Such a configurationcould replace or be used in combination with a fiber optic sensor.

Such a 3D camera can be used to detect when too many or too few itemswere dispensed. When this happens, an error procedure can be followed tocorrect the issue.

A 3D camera could be used to detect if an item did not enter the LFB.When such an event occurs, an error procedure can again be followed tofix the issue.

A 3D camera can be used to detect which product item moves from one rowinto another, when a row is being dispensed. When this movement betweentray rows occurs, error procedure can be followed to correct theproblem.

A 3D camera can be used to detect when an item is missing from the traythat should otherwise be there. Again, an error procedure can beinitiated to correct the issue.

FIG. 114 is a simplified block diagram showing inputs to a computersystem 11400 according to an embodiment comprising a processor 11402 anda non-transitory computer readable storage medium 11404.

As indicated, the processor is configured to receive one or more inputs11406. These inputs can include information received from a dispensestation, from a traveler, from camera(s), from a tray handlingapparatus, from storage, or from any other component of the producthandling system.

In particular, input from the cameras can comprise images that indicatethe position of the traveler, and the makeup of the goods inside. Camerainput can form a basis for validation of how many items were dispensedproperly.

In response to receiving such input(s), the processor is configured toreference 11407 additional information located in the non-transitorycomputer readable storage medium. Such additional information may bedata stored in a database 11408, with any of the pieces of datadiscussed herein being possible candidates for storage in the database.

The computer-readable storage medium may also store instructions forperforming certain specialized procedures according to various programs.Examples of such procedures can include but are not limited to one ormore of:

image processing 11420;

object recognition 11422;

container packing 11424;

tray assembly 11426;

ripeness prediction 11428;

physics simulation 11434;

path finding 11430;

(machine, reinforcement) learning 11432; and

others as have been described herein.

Based upon this input and referenced stored data, the processor isconfigured to perform a large variety of calculations to produce output11410 of different types. That output may be communicated to control oneor more of the various components of the product handling system,including but not limited to the dispense station, storage, trayhandling, intake, inspection, tray handling, and the others as have beendescribed herein.

It is emphasized that the block diagram of FIG. 114 is highly simplifiedfor purposes of illustration. That is, not each and every procedure orpiece of data may all be stored on a single non-transitory computerreadable storage medium. Rather, information storage may be distributed(e.g., locally in a system component, centrally on site, and/or in thecloud) amongst a plurality of non-transitory computer readable storagemedia for purposes of speed of access and efficient utilization ofavailable storage resources.

Moreover, a single processor may not be responsible for performing eachand every calculation depicted. Rather processing may be distributed (inseries, in parallel, or in some combination) amongst a plurality ofprocessors for purposes of speed and efficient utilization of availablecomputing resources.

In summary, embodiments for dispensing an item from a tray may utilize adepth camera to take a picture to find fiducials on a rim of a container(e.g., box, bag, platform) of a LFR. This image locates the LFR andcontainer relative to the camera and the dispense conveyors.

The image may also inspect the heights of the items in the container.The image may allow locating an optimal dispense location, such that abox can be drawn around that dispense location (e.g., as shown anddescribed above in connection with FIGS. 107A-B).

Then, the LFR moves along the floor to position the dispense location tobe in line with that dispense location box. The dispense conveyorsextend in and out, relative to the container of the LFR, depending uponwhere the dispense location box is. If the dispense location box is inthe back of the bag, the conveyors may extend out further.

A height of the container borne by the LFR may also adjust dependingupon the object. If fragile items have already been dispensed to thecontainer, and/or the item to be dispensed is fragile, then the LFR maymove the container to a greater height to reduce a distance that thedispensed item will drop, and hence any impact force.

Where the item to be dispensed, and/or existing items already dispensedto the container are not fragile, then the dispensing may just drop theitem to the correct location in an expedited manner. Thus certainapproaches may direct packing of non-fragile items first, followed bypacking of fragile items.

Additional details regarding possible LFB designs according to variousembodiments, are now described. Briefly, the LFB is a device that holdsa compliant bag from the top opening and supports it on a movingplatform from below.

FIG. 108 is a simplified perspective view of a LFB 10800 according to anembodiment. The LFB comprises a moving platform 10802 (which may includeforce sensors), supporting a compliant bag 10804.

The upper portion of the LFB in FIG. 108 may include a bag grab andpreload Mechanism 10806. A sensor 10808 in the upper portion may be usedto identify overflow of items in the bag.

The LFB of FIG. 108 may also include a moving base 10810. An embodimentof such a moving base is illustrated in FIG. 109

The platform height adjusts responding to commands and the volume ofitems inside of it and the desired space for the next item. The platformcan have force sensors to determine bag weight

The LFB device can have sensors on the bag opening to determine itemlevel in the bag. The LFB device can be either stationary, or have amotorized base.

FIG. 109 shows a simplified overhead view of a moving base according toan embodiment. The mechanics of the particular moving base of FIG. 109include a line sensor 10811. However, this is not required and otherguidance approaches can be utilized.

In particular, a motorized version of a LFB (or LFT) can guide itselfaround a physical space utilizing one or more of the following:

lines in the floor;

markers (e.g., RFID/barcode/other markers) positioned in the facility;and

active radar/LiDAR.

According to certain embodiments, upper portions of a LFB device canfeature an automated mechanism to offer bags to a human or to anotherautomated mechanism. In some embodiments, an upper portion of a LFBdevice can feature an automated mechanism to grab bags from an automatefeeder, and/or to preload the top of the bag in an open state.

Further details regarding a LFT design according to various particularembodiments, are now described. In particular, a LFT is a device thatholds multiple trays separable from the device.

FIG. 110 shows various views of LFT 11000 including a tray stack 11002according to an embodiment. The LFT may include a motorized base 11004,similar in many respects to a motorized based of a LFB.

The LFT further includes a lifting platform mechanism 11006 to lift thetray stack. The LFT may additionally comprise proximity and load sensorsto determine a presence and a mass of a lifted object. Sensors of theLFT may be used to determine a level and a height of the tray stack.

Also included in FIG. 110 is a side view illustrating a rod 11008 withhooks that grabs on to the trays. This element may be actuated by a trayrover.

A LFT device may include a locking or retention feature that preventstrays from popping out. The LFT device may include sensors/markers thatallow alignment to a specific tray level. The LFT device can stand onits own or be carried by a motorized unit.

When lifted by a motorized unit a mechanism may lock the tray stack tothe motorized unit. FIG. 111 shows a simplified view of a lockingfeature.

The LFT device may have features in the bottom and/or side that allowkinematic alignment to a mating component. FIG. 112 is a simplifiedfront view illustrating an alignment feature between the LFT and aplatform. This embodiment features cup cones with a locking mechanism.

Further details regarding a LFP design according to various particularembodiments, are now described. In particular, some items may be toolarge to big to fit in a grocery bag. Examples can include but are notlimited to 24 packs of water, 12 packs of paper towels, and firewood.

According to embodiments, such items can be stored in a bulk carousel orfixed shelving, and then dispensed using the same pick station tool.However, instead of dispensing into a bag, they could be dispensed ontoa platform.

In particular, the LFP comprises a line following robot having aplatform on top that can move cases or bulk goods dispensed from acarousel or fixed shelving. In some embodiments this platform wouldmatch the pick station height. However, the platform could also lowerfor easier packing into a delivery vehicle.

According to certain embodiments a platform could have conveyorsbuilt-in, to aid in dispensing items onto the platform. The platformcould alternatively just be a smooth surface that items slide onto.According to this approach, the handling of even large product items canbe automated.

Further details regarding a Line Following Mini-Bag (LFM) designaccording to embodiments, is now provided. In particular, the LFM is asubset of the LFB design which has some number of bags installed open,on a frame.

The LFM robot would travel around to receive dispensed specific SKUsbetter suited to ship in plastic bags. Examples of SKUs could includebut are not limited to, wet or dirty produce, bulk goods like nuts orgranola, and meats that could drip juices upon other items. The LFMrobot would travel to the storage bay to collect the dispensed items,and then travel to the delivery area to be consolidated with the rest ofthe order.

It is noted that a LFT design may include various inspectioncapabilities. An alternative to inspecting trays (mid-storage life), isto pull them out of the dispensing stations and onto the LFT.

The LFT could have an inspection camera present. That camera couldinspect the quality or ripeness of the items, and then a processor coulddecide how to handle those items.

As has been previously described, the storage and handling of productitems under climate controlled conditions, can be desirable. It is notedthat embodiments of either of/both the LFB and the LFT could includeon-board environmental control. This would allow maintenance ofcontrolled climate conditions throughout the fulfilment process.

Certain embodiments could provide for a cold material (e.g., an ice packor other chemical cooler) to be incorporated into the structure. Such aLFB and/or LFT could be stored in a cold area when not in use, and thenmove out to fulfil orders as needed.

According to some embodiment, LFT processes could be performed inclimate-controlled (e.g., cold) areas, and LFTs could have cold waitingareas as needed, before they have the opportunity to place the traysinto storage (e.g., fixed, or moveable carousels). Similarly LFBs couldmove into a climate controlled holding area after they have finishedfilling an order, but prior to arrival of the transport vehicle (e.g.,customer or delivery van).

Still another possible design which could be employed in connection withitem handling and delivery, is the LFBX. The LFBX design may comprise aline following robot featuring a box on top. This box could be any sizeor shape.

The box of the LFBX is specifically designed to interact with thedispensing system. This facilitates other item packing methods that arebetter suited for specific product types not otherwise working with aLFB design.

Clause 1K. An apparatus comprising:

fixed storage shelving maintained under climate controlled conditions;

a tray supported by the fixed storage shelving and bearing an itemhaving a dimension;

an elevator moveable in a vertical direction from a first heightproximate to the fixed storage shelving, to a second height proximate toa dispense mechanism; and

an element configured to move the tray from the fixed storage shelvingto the elevator,

wherein the dispense mechanism comprises,

-   -   a first conveyor belt configured to project through the tray to        contact the item at a first point, and    -   a second conveyor belt moveable relative to the first conveyor        belt and configured to project through the tray to contact the        item at a second point.

Clause 2K. An apparatus as in clause 1K wherein the second height isoverlying the dispense station, such that lowering of the elevatorcauses the first conveyor belt and the second conveyor belt to projectthrough the tray.

Clause 3K. An apparatus as in clause 1K wherein the second height isadjacent to the dispense mechanism.

Clause 4K. An apparatus as in clause 3K wherein the element is furtherconfigured to move the tray from the elevator to the dispense mechanismin order to allow the first conveyor belt and the second conveyor beltto project through the tray.

Clause 5K. An apparatus as in clause 1K wherein the element comprises afirst hook configured to engage the tray and move the tray in a firstdirection.

Clause 6K. An apparatus as in clause 5K wherein the tray furthercomprises a pin engaged by the first hook.

Clause 7K. An apparatus as in clause 5K wherein the element furthercomprises a second hook configured to engage the tray and move the traya second direction opposite to the first direction.

Clause 8K. An apparatus as in clause 7K wherein the first hook and thesecond hook are connected and driven together in the first direction andin the second direction.

Clause 9K. An apparatus as in clause 1K wherein both the first conveyorbelt and the second conveyor belt are configured to extend in the firstdirection prior to dispensing the item.

Clause 10K. An apparatus as in clause 9K further comprising a travelerconfigured to receive the item from the dispense mechanism and transportthe item for delivery to a vehicle.

Clause 11K. An apparatus as in clause 10K further wherein a bag of thetraveler receives the item.

Clause 12K. An apparatus as in clause 1K wherein the fixed storageshelving comprises an insulating panel and defines a hole through whichair is flowed.

Clause 13K. An apparatus as in clause 1K wherein the tray defines afirst slot through which the first conveyor belt projects, and defines asecond slot through which the second conveyor belt projects.

Clause 14K. An apparatus as in clause 1K wherein the elevator isdisposed within a shaft, the apparatus further comprising:

an idler located at a top of the shaft above the first height; and

a motor located at a bottom of the shaft below the second height.

Clause 15K. An apparatus as in clause 1K wherein the second conveyorbelt is moveable relative to the first conveyor belt utilizing a timingbelt.

Clause 16K. An apparatus as in clause 15K wherein the timing belt isfixed and a motor moves with the second conveyor belt along the timingbelt.

Clause 17K. An apparatus as in clause 15K wherein the second conveyorbelt moves with the timing belt.

Clause 18K. An apparatus as in clause 1K wherein the second conveyorbelt is moved relative to the first conveyor belt according to thedimension stored in a non-transitory computer readable storage medium.

Clause 19K. An apparatus as in clause 1K wherein the tray comprises aplurality of injection molded parts assembled by a machine.

Clause 20K. An apparatus as in clause 19K wherein the machine referencesthe dimension stored in a non-transitory computer readable storagemedium in order to assemble the plurality of injection molded parts intothe tray.

Clause 1L. An apparatus comprising:

a first conveyor belt configured to project through a tray to contact anitem at a first point; and

a second conveyor belt moveable relative to the first conveyor belt andconfigured to project through the tray to contact the item at a secondpoint,

wherein after a vertical position of the tray relative to the firstconveyor belt and to the second conveyor belt is changed, the firstconveyor belt and the second conveyor belt are configured to dispensethe item from the tray in a first direction.

Clause 2L. An apparatus as in clause 1L further comprising:

a first element configured to move the first conveyor belt in a verticaldirection to change the vertical position of the first conveyor beltrelative to the tray; and

a second element configured to move the second carriage in the verticaldirection to change the vertical position of the second conveyor beltrelative to the tray.

Clause 3L. An apparatus as in clause 1L further comprising an elementconfigured to move the tray in a vertical direction to change thevertical position of the tray relative to the first conveyor belt and tothe second conveyor belt.

Clause 4L. An apparatus as in clause 3L wherein the element comprises anelevator.

Clause 5L. An apparatus as in clause 4L wherein the elevator isvertically moveable to receive the tray from controlled climate storage.

Clause 6L. An apparatus as in clause 5L wherein the controlled climatestorage comprises a fixed shelf.

Clause 7L. An apparatus as in clause 1L wherein both the first conveyorbelt and the second conveyor belt are configured to extend in the firstdirection prior to dispensing the item.

Clause 8L. An apparatus as in clause 1L wherein both the first conveyorbelt and the second conveyor belt are configured to dispense the item bydropping into a bag.

Clause 9L. An apparatus as in clause 1L further comprising a thirdconveyor belt and a fourth conveyor belt configured to:

receive from the tray, another item conveyed by the first conveyor beltand the second conveyor belt in a second direction opposite to the firstdirection; and

after the item is dispensed, return the another item to the tray.

Clause 10L. An apparatus as in clause 1L further comprising a motorcausing the second conveyor belt to move relative to the first conveyorbelt using a timing belt.

Clause 11L. An apparatus as in clause 10L wherein the motor and thesecond conveyor belt move along the timing belt.

Clause 12L. An apparatus as in clause 10L wherein a location of themotor is fixed and the second conveyor belt moves with the timing belt.

Clause 13L. An apparatus as in clause 1L further comprising an elementconfigured to remove the tray from a storage shelf prior to the firstconveyor belt and the second conveyor belt projecting through the tray.

Clause 14L. An apparatus as in clause 13L wherein the element comprisesa hook.

Clause 15L. An apparatus as in clause 13L wherein:

the storage shelving is fixed; and

the element is configured to move in a vertical direction together withanother element that changes the vertical position of the tray relativeto the first conveyor belt and to the second conveyor belt.

Clause 16L. An apparatus as in clause 15L wherein the another elementcomprises an elevator.

Clause 17L. An apparatus as in clause 13L wherein the storage shelvingis part of moveable carousel.

Clause 18L. An apparatus as in clause 1L wherein:

a dimension of the item is stored in a non-transitory computer readablestorage medium, and the second conveyor belt is moved relative to thefirst conveyor belt based upon the dimension.

Clause 19L. An apparatus as in clause 18L wherein the tray comprises aplurality of injection molded parts assembled by a machine.

Clause 20L. An apparatus as in clause 19L wherein the machine referencesthe dimension to assemble the plurality of injection molded parts intothe tray.

Clause 1M. An apparatus comprising:

a climate-controlled storage location;

a tray supporting an item in the climate-controlled storage location;

a dispense station configured to receive the tray supporting the itemfrom the climate-controlled storage location;

a mobile robot configured to receive the item dispensed from the tray bythe dispense station;

a processor configured to control operation of the dispense station; and

a first camera in optical communication with the dispense station andthe mobile robot to send an image of the dispense station and the mobilerobot to the processor.

Clause 2M. An apparatus as in clause 1M wherein if the image indicatesthe item has not been successfully dispensed to the mobile robot, theprocessor is configured to instruct the dispense station to takecorrective action.

Clause 3M. An apparatus as in clause 1M wherein the processor determinesa specific location of the traveler to receive the item based upon theimage.

Clause 4M. An apparatus as in clause 3M wherein the specific location iswithin a bag, within a box, or upon a platform.

Clause 5M. An apparatus as in clause 3M wherein the traveler alreadyincludes other items.

Clause 6M. An apparatus as in clause 3M wherein the processor isconfigured to determine the specific location based upon a packingprocedure.

Clause 7M. An apparatus as in clause 6M wherein the packing procedureconsiders one or more of the following:

item packing density within the traveler;

item damage;

successful dispensing;

physics simulation; and

reinforcement learning.

Clause 8M. An apparatus as in clause 1M wherein the first camera isstereoscopic.

Clause 9M. An apparatus as in clause 1M wherein the image is a heatimage.

Clause 10M. An apparatus as in clause 1M wherein the processor isconfigured to compare the image with another image of the tray includingthe item, taken by a second camera prior to the tray moving to thedispense station.

Clause 11M. An apparatus as in clause 10M wherein the processorcomparing the image with the another image indicates that the item hasshifted position on the tray.

What is claimed is:
 1. An apparatus comprising: a tray configured tosupport an item, wherein the tray comprises at least one opening formedin a bottom of the tray; a dispense station comprising a pair ofconveyor belts configured to: project through the at least one openingof the tray; contact, while protecting through the tray, the itemsupported by the tray; and convey the item in a forward direction; atraveler oriented adjacent to the dispense station in the forwarddirection, the traveler comprising: a horizontally moveable basesupporting a vertically moveable platform, wherein the verticallymoveable platform is configured to support a container; and a processorconfigured to execute a program to: direct the traveler to movehorizontally to position the container at a first dispense location infront of the pair of conveyor belts; direct the pair of conveyor beltsto convey the item forward to the first dispense location such that thecontainer receives the item; and direct the traveler to again movehorizontally with the container and the received item to a seconddispense location different from the first dispense location.
 2. Theapparatus of claim 1, further comprising: a camera configured to capturea first image of the dispense station and the container, wherein inresponse to receiving the first image, the processor is configured toexecute the program to direct the camera to capture a second image ofthe item received within the container; and wherein in response to areceiving the second image, the processor is configured to execute theprogram to direct the traveler to move to the second dispense location.3. The apparatus of claim 2, wherein: in response to the receiving ofthe first image, the processor is configured to further direct thevertically moveable platform and the container to move to a firstdispense height; and in response to the receiving of the second image,the processor is configured to further direct the vertically moveableplatform and the container to move to a second dispense height that islower than the first dispense height.
 4. The apparatus of claim 1,wherein the container comprises a compliant bag, a box, or a platform.5. The apparatus of claim 1, wherein the pair of conveyor beltscomprises: a first conveyor belt configured to project through a firstopening of the tray to contact the item at a first point; and a secondconveyor belt moveable relative to the first conveyor belt andconfigured to project through a second opening of the tray to contactthe item at a second point, wherein the at least one opening comprisesthe first opening and the second opening.
 6. The apparatus of claim 5,further comprising: a first actuator configured to move the firstconveyor belt in a vertical direction to change a vertical position ofthe first conveyor belt relative to the tray prior to conveying the itemforward; and a second actuator configured to move the second conveyorbelt in the vertical direction to change a vertical position of thesecond conveyor belt relative to the tray prior to conveying the itemforward.
 7. The apparatus of claim 5, further comprising: an actuatorconfigured to move the tray in a vertical direction to change a verticalposition of the tray (a) relative to the first conveyor belt and (b)relative to the second conveyor belt, prior to conveying the itemforward.
 8. The apparatus of claim 7, wherein the actuator comprises amoveable elevator.
 9. The apparatus of claim 8, wherein the moveableelevator is vertically moveable to receive the tray from a controlledclimate storage unit.
 10. The apparatus of claim 9, wherein thecontrolled climate storage unit comprises a fixed shelf.
 11. Theapparatus of claim 5, wherein both the first conveyor belt and thesecond conveyor belt are configured to extend in the forward directionprior to dispensing the item at the first dispense location.
 12. Theapparatus of claim 5, wherein both the first conveyor belt and thesecond conveyor belt are configured to dispense the item at the firstdispense location by permitting the item to drop into the container. 13.The apparatus of claim 5, further comprising: a third conveyor belt anda fourth conveyor belt each configured to: receive from the tray,another item conveyed by the first conveyor belt and the second conveyorbelt in a backward direction opposite to the forward direction; andreturn the another item to the tray after the item is dispensed at thefirst dispense location.
 14. The apparatus of claim 5, furthercomprising: a motor configured to move the second conveyor belt relativeto the first conveyor belt using a timing belt.
 15. The apparatus ofclaim 14, wherein the motor and the second conveyor belt are configuredto move along the timing belt.
 16. The apparatus of claim 14, wherein alocation of the motor is fixed, and the second conveyor belt isconfigured to move with the timing belt.
 17. The apparatus of claim 5,further comprising: a tray removal mechanism configured to remove thetray from a storage shelf prior to the first conveyor belt and thesecond conveyor belt projecting through the first and second openings ofthe tray.
 18. The apparatus of claim 17, wherein the tray removalmechanism comprises a hook.
 19. The apparatus of claim 17, wherein: thestorage shelf is fixed; and the tray removal mechanism is configured tomove vertically to change a vertical position of the tray relative tothe first conveyor belt and to the second conveyor belt.
 20. Theapparatus as in claim 17, wherein the storage shelf is part of amoveable carousel.
 21. The apparatus of claim 1, wherein the traycomprises a plurality of injection molded parts.